DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 101
U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 17 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter because it is a claim to a computer program product.
Dependent claims not mentioned specifically above inherit the deficiencies from the claims on which they depend.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 – 20 are rejected under 35 U.S.C. 103 as being unpatentable over Vock et al. (Patent: 6093923 A) in view of Bouchard et al. (Publication: US 2021/0213351 A1).
Regarding claim 1,Vock discloses an augmented reality simulation method, wherein the method is applied to an device, and the method comprises (Column 6 line 33 - FIG. 9 illustrates a system, constructed according to the invention, for tracking golf ball motion with an analog camera; the memory stores the instructors and processed by the processor to perform the methods ):
obtaining, by the Device, a real-world scene and physical information associated with a target object in the real-world scene (Column 15 lines 19 - hit golf balls 304 into the driving range 300. Items located in the driving range 300 include water hazards 306b and sand traps 306a, pin targets 308, and ball retrieval trucks 310. The data acquisition subsystem 312 collects scene imagery from the driving range 300, converts it to electrical energy, and then processes the resulting electrical signals so as to identify and locate all balls 304 moving within the driving range 300. The support structure 314 physically holds the subsystem 312 in a position where it can view, preferably, the full driving range 300 (alternatively, if the subsystem 312 does not view the entire range 300, additional subsystems 312 can be installed to view the range 100, if desired, so as to image the entire range 300).), and
determining first operation information of the target object based on the physical information associated with the target object, wherein the first operation information indicates a physical operation to be performed on the target object (Column 14 line 44 - After the ball is hit, the system of the invention tracks movement such as shown in FIG. 8A. Note that for successive frames, here shown as single pixel images 200 of the ball 202, the system tracks the overall motion of the ball 202, which in general follows an arc-like pattern 204 through the range 199. As such, it is without excessive processing that the system of the invention can determine overall distance "D" to the final destination 206 of the ball 202 even though the ball 202 hit the net 208. That is, the arc 204 is curve fit to determine the distance "D" hit by the appropriate user at tee-off 210. This determination can occur at any time, provided that there is enough frame data to make such an assessment. For example, the arc 204 determined by the time the ball 202 reaches position 212 is probably sufficient to determine overall distance "D" yet to be traveled. Accordingly, a user can be notified of the expected travel distance "D" before the ball actually reaches that location 206.
); and
generating, by the Device, an Scene based on the real-world scene and the first operation information, wherein the Scene comprises the first operation information (Column 14 line 24 - FIG. 8 illustrates one embodiment of determining distance of a golf ball, in accord with the invention; and specifically illustrates a frame 190 of image data. For ease of illustration, FIG. 8 is shown schematically as a top view in two dimensions; when, in reality, the view of the camera onto the range is skewed, or perspective, and thus so is the frame 190 of image data. In FIG. 8, the tee-off position 192 is imaged to pixels "A," "B" and "C" of the associated camera system (e.g., the camera system 100 of FIG. 5). Thus, when the system of the invention "sees" motion in the pixels "A," "B" and "C" it knows that the track of a golf ball originating from that location belongs to the associated user; and that the final distance measured for the golf ball 194 hit into the range 196 should be sent to that location 192 and the associated LCD. The pixels A,B,C relative to the physical dimensions of the tee-off position 192 and ball 194 are not to scale. Note that the frame 190 preferably includes the tee-off position 192, the range ground 196, and the net 198, which means that the associated camera (not shown) should view that entire field of regard.
Fig 9 display generated digital information like images, sound, or 3D models directly onto your view of the physical world, “Augmented reality”.).
Vock does not disclose; however Bouchard discloses
an augmented reality (AR) device ([0096] - provide for detailed analysis of a game within a virtual 3-D grid and represent game analytics to viewers via virtual reality or augmented reality displays);
AR device ([0096] - provide for detailed analysis of a game within a virtual 3-D grid and represent game analytics to viewers via virtual reality or augmented reality displays);
AR scene ([0096] - provide for detailed analysis of a game within a virtual 3-D grid and represent game analytics to viewers via virtual reality or augmented reality displays).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Vock with an augmented reality (AR) device; AR device; AR scene as taught by Bouchard . The motivation for doing is to enable better understanding of the ball trajectory thus enhance user experience.
Regarding claim 2, Vock in view of Bouchard disclose all the limitation of claim 1.
Vock discloses determining, by the Device, N pieces of expected motion information based on the physical information associated with the target object and N pieces of candidate operation information, wherein the N pieces of candidate operation information are in one-to-one correspondence with the N pieces of expected motion information, and N is a positive integer greater than 1 (
Column 14 line 44 - After the ball is hit, the system of the invention tracks movement such as shown in FIG. 8A. Note that for successive frames, here shown as single pixel images 200 of the ball 202, the system tracks the overall motion of the ball 202, which in general follows an arc-like pattern 204 through the range 199. As such, it is without excessive processing that the system of the invention can determine overall distance "D" to the final destination 206 of the ball 202 even though the ball 202 hit the net 208. That is, the arc 204 is curve fit to determine the distance "D" hit by the appropriate user at tee-off 210. This determination can occur at any time, provided that there is enough frame data to make such an assessment. For example, the arc 204 determined by the time the ball 202 reaches position 212 is probably sufficient to determine overall distance "D" yet to be traveled. Accordingly, a user can be notified of the expected travel distance "D" before the ball actually reaches that location 206.
Column 14 line 24 - FIG. 8 illustrates one embodiment of determining distance of a golf ball, in accord with the invention; and specifically illustrates a frame 190 of image data. For ease of illustration, FIG. 8 is shown schematically as a top view in two dimensions; when, in reality, the view of the camera onto the range is skewed, or perspective, and thus so is the frame 190 of image data. In FIG. 8, the tee-off position 192 is imaged to pixels "A," "B" and "C" of the associated camera system (e.g., the camera system 100 of FIG. 5). Thus, when the system of the invention "sees" motion in the pixels "A," "B" and "C" it knows that the track of a golf ball originating from that location belongs to the associated user; and that the final distance measured for the golf ball 194 hit into the range 196 should be sent to that location 192 and the associated LCD. The pixels A,B,C relative to the physical dimensions of the tee-off position 192 and ball 194 are not to scale. Note that the frame 190 preferably includes the tee-off position 192, the range ground 196, and the net 198, which means that the associated camera (not shown) should view that entire field of regard.) ; and
selecting, by the Device from the N pieces of candidate operation information, corresponding candidate operation information of which expected motion information meets a specified condition as the first operation information (
Column 14 line 44 - After the ball is hit, the system of the invention tracks movement such as shown in FIG. 8A. Note that for successive frames, here shown as single pixel images 200 of the ball 202, the system tracks the overall motion of the ball 202, which in general follows an arc-like pattern 204 through the range 199. As such, it is without excessive processing that the system of the invention can determine overall distance "D" to the final destination 206 of the ball 202 even though the ball 202 hit the net 208. That is, the arc 204 is curve fit to determine the distance "D" hit by the appropriate user at tee-off 210. This determination can occur at any time, provided that there is enough frame data to make such an assessment. For example, the arc 204 determined by the time the ball 202 reaches position 212 is probably sufficient to determine overall distance "D" yet to be traveled. Accordingly, a user can be notified of the expected travel distance "D" before the ball actually reaches that location 206.
Column 14 line 24 - FIG. 8 illustrates one embodiment of determining distance of a golf ball, in accord with the invention; and specifically illustrates a frame 190 of image data. For ease of illustration, FIG. 8 is shown schematically as a top view in two dimensions; when, in reality, the view of the camera onto the range is skewed, or perspective, and thus so is the frame 190 of image data. In FIG. 8, the tee-off position 192 is imaged to pixels "A," "B" and "C" of the associated camera system (e.g., the camera system 100 of FIG. 5). Thus, when the system of the invention "sees" motion in the pixels "A," "B" and "C" it knows that the track of a golf ball originating from that location belongs to the associated user; and that the final distance measured for the golf ball 194 hit into the range 196 should be sent to that location 192 and the associated LCD. The pixels A,B,C relative to the physical dimensions of the tee-off position 192 and ball 194 are not to scale. Note that the frame 190 preferably includes the tee-off position 192, the range ground 196, and the net 198, which means that the associated camera (not shown) should view that entire field of regard.).
Regarding claim 3, Vock in view of Bouchard disclose all the limitation of claim 2.
Vock discloses wherein the specified condition comprises one or more of the following: an expected score obtained through single motion of the target object performed based on expected motion information is the highest; an expected location at which the target object stays after completing single motion based on the expected motion information is within a specified space range; and a quantity of violated motion rules in single motion of the target object performed based on the expected motion information is the smallest (
Column 19 line 60 - The track correlator 382 receives the target data for each camera frame from the camera subsystem 312 via the system bus 318. Each target is compared to the expected positions and amplitudes for the current track files. If a target is found to fall within the expected position and amplitude of a track file, then a new record is added to the track file for that target and a trajectory is calculated using a spline or least squares fit algorithm, for example. Next, a confidence level for that ball's trajectory is estimated by calculating the RMS error between the ball trajectory and ball position with the most current positions weighted to a higher degree than older positions. The trajectory is then extended to predict the ball's position and amplitude for the next frame. Based on the confidence level, a range of expected amplitudes and positions, centered about the predicted amplitude and position, are formed and the old estimates in the track file 380 are overwritten.
Column 14 line 44 - After the ball is hit, the system of the invention tracks movement such as shown in FIG. 8A. Note that for successive frames, here shown as single pixel images 200 of the ball 202, the system tracks the overall motion of the ball 202, which in general follows an arc-like pattern 204 through the range 199. As such, it is without excessive processing that the system of the invention can determine overall distance "D" to the final destination 206 of the ball 202 even though the ball 202 hit the net 208. That is, the arc 204 is curve fit to determine the distance "D" hit by the appropriate user at tee-off 210. This determination can occur at any time, provided that there is enough frame data to make such an assessment. For example, the arc 204 determined by the time the ball 202 reaches position 212 is probably sufficient to determine overall distance "D" yet to be traveled. Accordingly, a user can be notified of the expected travel distance "D" before the ball actually reaches that location 206.).
Regarding claim 4, Vock in view of Bouchard disclose all the limitation of claim 1.
Vock discloses obtaining, by the Device, expected motion information input by a user; and determining, by the Device, the first operation information based on the physical information associated with the target object and the expected motion information input by the user (
Fig 9 display generated digital information like images, sound, or 3D models directly onto your view of the physical world, “Augmented reality”.
Column 30 line 50 - determining a distance traveled beyond a fairway net by estimating golf ball motion along an arc, the arc being determined by discrete points representing golf ball detections through a plurality of frames.
Column 14 line 44 - After the ball is hit, the system of the invention tracks movement such as shown in FIG. 8A. Note that for successive frames, here shown as single pixel images 200 of the ball 202, the system tracks the overall motion of the ball 202, which in general follows an arc-like pattern 204 through the range 199. As such, it is without excessive processing that the system of the invention can determine overall distance "D" to the final destination 206 of the ball 202 even though the ball 202 hit the net 208. That is, the arc 204 is curve fit to determine the distance "D" hit by the appropriate user at tee-off 210. This determination can occur at any time, provided that there is enough frame data to make such an assessment. For example, the arc 204 determined by the time the ball 202 reaches position 212 is probably sufficient to determine overall distance "D" yet to be traveled. Accordingly, a user can be notified of the expected travel distance "D" before the ball actually reaches that location 206.)
Regarding claim 5, Vock in view of Bouchard disclose all the limitation of claim 1.
Vock discloses generating, by the Device, the Scene based on the real-world scene, the first operation information, and the expected motion information corresponding to the first operation information, wherein the Scene further comprises the expected motion information corresponding to the first operation information (
Fig 9 display generated digital information like images, sound, or 3D models directly onto your view of the physical world, “Augmented reality”.
Column 30 line 50 - determining a distance traveled beyond a fairway net by estimating golf ball motion along an arc, the arc being determined by discrete points representing golf ball detections through a plurality of frames.
Column 14 line 44 - After the ball is hit, the system of the invention tracks movement such as shown in FIG. 8A. Note that for successive frames, here shown as single pixel images 200 of the ball 202, the system tracks the overall motion of the ball 202, which in general follows an arc-like pattern 204 through the range 199. As such, it is without excessive processing that the system of the invention can determine overall distance "D" to the final destination 206 of the ball 202 even though the ball 202 hit the net 208. That is, the arc 204 is curve fit to determine the distance "D" hit by the appropriate user at tee-off 210. This determination can occur at any time, provided that there is enough frame data to make such an assessment. For example, the arc 204 determined by the time the ball 202 reaches position 212 is probably sufficient to determine overall distance "D" yet to be traveled. Accordingly, a user can be notified of the expected travel distance "D" before the ball actually reaches that location 206.
Column 15 line 18 - FIG. 10 illustrates another embodiment of the invention as installed at a driving range 300. The golf players at one or more tee-off locations 302 hit golf balls 304 into the driving range 300. Items located in the driving range 300 include water hazards 306b and sand traps 306a, pin targets 308, and ball retrieval trucks 310. The data acquisition subsystem 312 collects scene imagery from the driving range 300, converts it to electrical energy, and then processes the resulting electrical signals so as to identify and locate all balls 304 moving within the driving range 300. The support structure 314 physically holds the subsystem 312 in a position where it can view, preferably, the full driving range 300 (alternatively, if the subsystem 312 does not view the entire range 300, additional subsystems 312 can be installed to view the range 100, if desired, so as to image the entire range 300, “based on the real-world scene”).
Regarding claim 6, Vock in view of Bouchard disclose all the limitation of claim 1.
Vock discloses discloses attribute information of the target object, force-bearing information of the target object, and attribute information of an object that the target object expects to touch ( Column 12 lin 50 - FIG. 6A illustrates the relationship between ball size and signal strength, in accord with the invention. A FPA 140 is illustratively shown in FIG. 6A as a representative solid state detector array; and a ball's motion is sequentially captured during image framing as images 142a, 142b, 142c and 142d. Note that the ball's overall image size, relative to the pixel IFOV dimensions 144, at positions 142a-142d changes during the flight. At images 142a, 142b and 142c, the ball subtends an angle that is smaller than the IFOV; and therefore the distancing functionality as to 1/R.sup.2 applies. At image 142e, the ball image is much larger than a given IFOV so that, for example, detectors 146 and 147 image the ball with the same approximate image strength. However, neither detector 146, 147 is used, alone, to determine distance. Therefore, the signal strengths from all of the detectors which image ball 142e are summed to determine a signal strength; and that signal strength can be used to determine distance.
Column 2 line 55 - The system includes a processing subsystem to determine how far a ball travels before the ball completes its journey. Specifically, this aspect includes the steps of monitoring the ball's movement during a first period of time, e.g., 1 second, and determining where the ball would land by completing the arc traveled by the ball before the ball actually travels that distance; and relaying this information to the user. This information can be updated, during flight and up until the ball lands, so that the user has a better and better determination of the ball's traveled distance. Once the ball lands, and/or comes to a stop (the system can notify the user of either occurrence, and distance, selectively), that selected distance is displayed to the user with great accuracy. ).
Regarding claim 7, Vock in view of Bouchard disclose all the limitation of claim 1.
Vock discloses obtaining, by the Device, second operation information indicating a physical operation actually performed by the user on the target object; and
adjusting, by the Device, the Scene based on the second operation information, wherein the Scene comprises the second operation information and/or third operation information, and the third operation information is obtained based on difference information between the second operation information and the first operation information (
Fig 9 obtaining generated digital information like images, sound, or 3D models directly onto your view of the physical world, “Augmented reality”.
Column 3 line 5 - The frames of image data taken at the camera are stored and processed by a frame grabber at preferably high rates. On-board processing within the card enables high speed calculations at frame rates suitable to capture golf ball movements around the driving range. By way of example, if a typical user hits a golf ball 200 yards in 5 seconds (i.e., 1440 inches per second), and if each pixel's instantaneous field of view (IFOV) is on the order of a golf ball at 200 yards, then 1440 Hz frame rates will capture a golf ball at each frame and at adjacent pixels, frame to frame. When the composite image is analyzed, such high speed image capture will provide a very smooth motion curve of the golf ball's travel, making distance determination relatively easy.
Column 13 line 19 - FIG. 6B illustrates that golf ball distance can also be determined geometrically because the ball’s size is a constant. In particular, if the focal plane 150 is sufficiently dense in IFOV pixels 150a, as compared to the geometric image of the ball over the range, such as illustratively shown by golf ball images 152, 154 (image 152 denoting a "closer" golf ball due to its larger image size; and image 154 denoting a "further" golf ball due to its smaller image size), then the distance to the ball can be determined by a ratio. That is, by calibrating the number of pixels which correspond to a ball's width at, for example, the tee-off position (being at a fixed, known distance from the camera), then subsequent frames readily specify golf ball distance, from the camera, by a ratio between the numbers of detectors which image the linear dimensions of the ball (e.g., if eight detectors image the ball at fifty yards; then sixteen detectors image the ball at twenty-five yards). ).
Regarding claim 8, Vock in view of Bouchard disclose all the limitation of claim 1.
Bouchard discloses wherein the real-world scene is a billiards scene, the target object is a target billiard ball, and the first operation information indicates at least one of the following: a to-be-hit point of a cue on the target billiard ball, a to-be-hit force of the cue for the target billiard ball, and a to-be-hit direction of the cue for the target billiard 2 ball ([0096], Fig. 6C - The image capturer E in conjunction with the controller C, can provide for detailed analysis of a game within a virtual 3-D grid and represent game analytics to viewers via virtual reality or augmented reality displays to further explain to viewers particular details regarding angles, speed and force of cue stick strikes on cue balls, strike region on cue balls, strike regions of object balls when hit by the cue balls, types of movements of the cue balls and object balls. These precise details are amassed by the controller C in its memory in order to provide relevant game statistics as well as to provide a database/databank for research by players, enthusiasts and coaches.) .
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Vock in view of Bouchard with wherein the real-world scene is a billiards scene, the target object is a target billiard ball, and the first operation information indicates at least one of the following: a to-be-hit point of a cue on the target billiard ball, a to-be-hit force of the cue for the target billiard ball, and a to-be-hit direction of the cue for the target billiard 2 ball as taught by Bouchard . The motivation for doing is to enable better understanding of the ball trajectory thus enhance user experience.
Regarding claim 9, see rejection on claim 1.
Regarding claim 10, see rejection on claim 2.
Regarding claim 11, see rejection on claim 3.
Regarding claim 12, see rejection on claim 4.
Regarding claim 13, see rejection on claim 5.
Regarding claim 14, see rejection on claim 6.
Regarding claim 15, see rejection on claim 7.
Regarding claim 16, see rejection on claim 8.
Regarding claim 17, see rejection on claim 1.
Regarding claim 18, see rejection on claim 2.
Regarding claim 19, see rejection on claim 3.
Regarding claim 20, see rejection on claim 4.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MING WU whose telephone number is (571)270-0724. The examiner can normally be reached on Monday - Thursday and alternate Fridays: 9:30am - 6:00pm EST .
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona Faulk can be reached on 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MING WU/
Primary Examiner, Art Unit 2618
Prosecution Insights