DETAILED ACTION
1. Claims 1-39 of application 18/955,133, filed on 21-November-2024, are presented for examination. The IDS received on the same date has been considered.
The present application, filed on or after 16-March-2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
2. Claim 32 is objected to under 37 CFR 1.75(d)(1), wherein --
The claim or claims must conform to the invention as set forth in the remainder of the specification and the terms and phrases used in the claims must find clear support or antecedent basis in the description so that the meaning of the terms in the claims may be ascertainable by reference to the description. (See 1.58(a))
The following is a list of the specific objections:
The phrase “information form the moving object” [ln. 5] is understood as and should be
changed to “information from the moving object”.
Claim Rejections under 35 U.S.C. § 102(a)(1)
3.1 The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
3.2.1 Claims 1, 26-27, 31-32 and 35-39 are rejected under 35 U.S.C. § 102(a)(1) as being anticipated by Iwane, USP Publication 2022/0294457.
3.2.2 Iwane discloses:
Claims 1 and 35-39: An information processing system including a moving object involving maneuvering [¶0085 (the present disclosure is also applicable to the control for automatic
driving following other vehicles)], and an image-capturable moving apparatus having an image capturing device mounted thereon which captures an image of the moving object [0081-0084 (an imaging system 800 relating to an in-vehicle camera.); Fig. 16C], comprising:
at least one processor [0082]; and
a memory coupled to the at least one processor storing instructions that, when executed by the processor, cause the processor to function as [0082 (The various processors described above may be implemented by specially designed hardware, or may be implemented by general-purpose hardware that performs operations based on software modules (which necessarily require a memory to be stored on).); 0088]:
an acquisition unit configured to acquire maneuvering instruction information [0082 (The imaging system 800 also includes a distance-obtaining portion 803, which is a processor that calculates the distance to an object based on the calculated disparity, a collision determination portion 804, which is a processor that determines whether there is a possibility of collision based on the calculated distance.)] of the moving object [0085 (the present disclosure is also applicable to the control for automatic driving following other vehicles)]; and
an image capturing control unit configured to control image capturing performed by the image capturing device [0081-0084 (an imaging system 800 relating to an in-vehicle camera.); Fig. 16C],
wherein the image capturing control unit controls image-capturing conditions at a time of image-capturing performed by the image capturing device, based on the maneuvering instruction information [0082 (The imaging system 800 includes a disparity-obtaining portion 802, which is a processor that calculates disparity (the phase difference in disparity images) from a plurality of pieces of image data obtained by the photoelectric conversion device 805…The disparity obtaining portion 802 and the distance-obtaining portion 803 are examples of information-obtaining means for obtaining information such as distance information with respect to an object. That is, the distance information relates to disparity, defocus amount, distance to the object, and the like. The collision determination portion 804 may determine a possibility of collision using any of these distance information pieces.); 0084 (the imaging system 800 captures images of the surroundings of the vehicle, for example, the front side or the rear side of the vehicle. FIG. 16C shows the imaging system 800 for capturing images of the front side of the vehicle (image range 850). The vehicle information obtaining device 810 sends an instruction to operate the imaging system 800 to capture an image.)].
Claim 26: wherein environmental information around the object involving maneuvering is acquired in addition to the maneuvering instruction information [0084 (the imaging system 800
captures images of the surroundings of the vehicle, for example, the front side or the rear side of the vehicle.)].
Claim 27: wherein the environmental information is information including at least one of position information of the object involving maneuvering [0085 (The example described above relates to the control for avoiding collision with other vehicles, but the present disclosure is also applicable to the control for automatic driving following other vehicles, the control for automatic driving that prevents deviation from the lane, and the like.)], a weather around the object, a map, a structure, sunny/shaded areas, and a time.
Claim 31: wherein in a case where it is difficult to acquire the maneuvering instruction information, the image capturing device controls the image-capturing conditions at the time of image-capturing based on at least image analysis information of the object involving maneuvering [0082 (The imaging system 800 includes a disparity-obtaining portion 802, which is a processor that calculates disparity (the phase difference in disparity images) from a plurality of pieces of image data obtained by the photoelectric conversion device 805…The disparity obtaining portion 802 and the distance-obtaining portion 803 are examples of information-obtaining means for obtaining information such as distance information with respect to an object. That is, the distance information relates to disparity, defocus amount, distance to the object, and the like. The collision determination portion 804 may determine a possibility of collision using any of these distance information pieces.);.
Claim 32: wherein the image capturing control unit is mounted on the image-capturable moving apparatus [0081-0084 (an imaging system 800 relating to an in-vehicle camera.); Fig. 16C], and wherein the acquisition unit acquires the maneuvering instruction information form the moving object by requesting the maneuvering instruction information to the moving object [0082 (The imaging system 800 also includes a distance-obtaining portion 803, which is a processor that calculates the distance to an object based on the calculated disparity, a collision determination portion 804, which is a processor that determines whether there is a possibility of collision based on the calculated distance.)].
3.3.1 Claims 1-8, 12-14, 17, 26-27, 31-32 and 34-39 are rejected under 35 U.S.C. § 102(a)(1) as being anticipated by Kearns et al, USP Publication 2016/0188977.
3.3.2 Kearns discloses:
Claims 1 and 35-39: An information processing system including a moving object involving maneuvering [¶0064 (In the example of patrolling, a mobile robot can make rounds within a facility to monitor activity and serve as a deterrence to potential trespassers. For tracking and following, the mobile robot can detect a presence of a person, track movement and predict trajectories of the person, follow the person as he/she moves)], and an image-capturable moving apparatus having an image capturing device mounted thereon which captures an image of the moving object [0064 (a mobile robot can make rounds within a facility to monitor activity and…capture images of the person and relay the captured images and other pertinent information (e.g., map location, trajectory, time stamp, text message, email communication, aural wireless communication, etc.) to a remote recipient.)], comprising:
at least one processor [0068-0069; 0101]; and
a memory coupled to the at least one processor storing instructions that, when executed by the processor, cause the processor to function as [0068-0069; 0101]:
an acquisition unit configured to acquire maneuvering instruction information of the moving object [0065 (a robot 100 patrolling an environment 10 may sense the presence of a person 20 within that environment 10 using one or more sensors, such as a proximity sensor 410 and/or an imagining sensor 450 of a sensor module 300 in communication with a controller system 500 (also referred to as a controller) of the robot 100. The robot 100 may maneuver to have the person 20 within a sensed volume of space S and/or to capture images 50 ( e.g., still images or video) of the person 20 using the imaging sensor 450.)]; and
an image capturing control unit configured to control image capturing performed by the image capturing device [0065 (a robot 100 patrolling an environment 10 may sense the presence of a person 20 within that environment 10 using one or more sensors, such as…an imagining sensor 450 of a sensor module 300 in communication with a controller system 500 (also referred to as a controller) of the robot 100. The robot 100 may maneuver to have the person 20 within a sensed volume of space S and/or to capture images 50 ( e.g., still images or video) of the person 20 using the imaging sensor 450.)],
wherein the image capturing control unit controls image-capturing conditions at a time of image-capturing performed by the image capturing device, based on the maneuvering instruction information [0067 (The controller 500 integrates the movements of the robot 100, the person 20, and the shutter speed and/or focal limitations of the imaging sensor 450 so that the robot 100 accelerates and decelerates to accommodate for the velocity of the person 20 and the shutter speed and/or focal limitations of the imaging sensor 450 while positioning itself to capture an image 50 ( e.g., take a picture) of the moving person 20. The controller 500 predicts the trajectory of the moving person and calculates the stop time and/or deceleration time of the robot 100 and the focal range and shutter speed of the imaging sensor 450 in deciding at which distance from the moving person 20 to capture a photograph or video clip.)].
Claim 2: wherein the processor is caused to further function as a shooting scenario selection unit configured to select a shooting scenario from a shooting scenario list in which conditions concerning image-capturing are described, and wherein the shooting scenario list describes a shooting start condition and a shooting end condition, which are determined based on at least the maneuvering instruction information, and the image-capturing conditions of the image capturing device [0067 (The controller 500 predicts the trajectory of the moving person and calculates the stop time and/or deceleration time of the robot 100 and the focal range and shutter speed of the imaging sensor 450 in deciding at which distance from the moving person 20 to capture a photograph or video clip. For instance, when the person 20 is running away from the robot 100, the controller system 500 may command the robot 100 to speed up ahead of the person 20 so that the person 20 is centered in the field of view 452 of the imaging sensor 450 once the robot 100 slows, stops and/or catches up to the person 20 for capturing a clear image 50 (i.e. a first choice in the list of appropriate actions). In other situations, the controller 500 may command the robot 100 to back away from the person 20 if the person 20 is determined to be too close to the imaging sensor 450 to capture a crisp image 50 (the second choice in the list of appropriate actions). Moreover, the controller 500 may command the robot 100 to follow the person 20, for example, at a distance, to observe the person 20 for a period of time (a third choice in the list of appropriate actions).)].
Claim 3: wherein the image-capturing conditions include at least one of a frame rate, a shutter speed [0066 (the imaging sensor 450 is a camera with a fast shutter speed that rapidly takes successive images 50 of one or more moving targets and batches the one or more images 50 for transmission.); 0067], and an exposure condition.
Claim 4: wherein the image capturing control unit controls the image capturing device to perform image capturing of the moving object involving maneuvering, according to the shooting start condition, the shooting end condition, and the image-capturing conditions, which are described in a shooting scenario selected by the shooting scenario selection unit [0067 (The controller 500 predicts the trajectory of the moving person and calculates the stop time and/or deceleration time of the robot 100 and the focal range and shutter speed of the imaging sensor 450 in deciding at which distance from the moving person 20 to capture a photograph or video clip. For instance, when the person 20 is running away from the robot 100, the controller system 500 may command the robot 100 to speed up ahead of the person 20 so that the person 20 is centered in the field of view 452 of the imaging sensor 450 once the robot 100 slows, stops and/or
catches up to the person 20 for capturing a clear image 50. In other situations, the controller 500 may command the robot 100 to back away from the person 20 if the person 20 is determined to be too close to the imaging sensor 450 to capture a crisp image 50. Moreover, the controller 500 may
command the robot 100 to follow the person 20, for example, at a distance, to observe the person 20 for a period of time.)].
Clam 5: wherein the maneuvering instruction information is information for instructing a position and an operation of the moving object [0065 (a robot 100 patrolling an environment 10 may sense the presence of a person 20 within that environment 10 using one or more sensors, such as a
proximity sensor 410 and/or an imagining sensor…The controller 500 may tag the image 50 with a
location and/or a time associated with capturing the image 50 of the person 20)].
Claim 6: wherein the maneuvering instruction information is information instructing at least one of a speed and an orientation of the movable object, an operation of a movable mechanism [0065 (The robot 100 may maneuver to have the person 20 within a sensed volume of space S and/or to capture images 50 ( e.g., still images or video) of the person 20 using the imaging
sensor 450.)], a notification using sound or light, and light emission from a light source [0093 (a sheet of light)].
Claim 7: wherein the processor is caused to further function as a scene attention-getting degree determination unit configured to determine a scene attention-getting degree of an image-capturing scene based on the maneuvering instruction information [0064 (relay the captured images and other pertinent information (e.g., map location, trajectory, time stamp, text message, email communication, aural wireless communication, etc.) to a remote recipient); 0065 (the robot 100 may send the surveillance report 1010 as an email, a text message, a short message service (SMS) message, or an automated voice mail over a network 102 to the remote security system 1000. Other types of messages are possible as well, which may or may not be sent using the network 102); 0069 (The controller 500 communicates with the security system 1000, which may transmit signals to the controller 500 indicating one or more alarms within the patrolling environment 10 and locations associated with the alarms.)].
Claim 8: wherein the image capturing control unit performs control to increase a frame rate in a case where it is determined that the scene attention-getting degree is high and performs control to reduce the frame rate in a case where it is determined that the scene attention-getting degree is low [0066 (the imaging sensor 450 is a camera with a fast shutter speed that rapidly takes successive images 50 of one or more moving targets and batches the one or more images 50 for transmission.); 0067 (The controller 500 may account for dynamics of the person 20 ( e.g., location, heading, trajectory, velocity, etc.), shutter speed of the imaging sensor 450 and dynamics of the robot 100 (e.g., velocity/holonomic motion) to aim the corresponding field of view 452 of the imaging sensor 450 to continuously perceive the person 20 within the field of view 452, so that the person 20 is centered in the captured image 50 and the image 50 is clear.)].
Claim 12: wherein the scene attention-getting degree determination unit acquires maneuvering instruction information of another moving object and determines the scene attention-getting degree by using the maneuvering instruction information of the other moving object in combination with the maneuvering instruction information [0067 (The controller system 500 may
execute movement commands to maneuver the robot 100 in relation to the location of the person 20…The movement commands may be based on a trajectory prediction TR and velocity of the person 20, in addition to dynamics of the robot 100 and/or shutter speed of the imaging sensor 450. The controller 500 integrates the movements of the robot 100, the person 20, and the shutter speed and/or focal limitations of the imaging sensor 450 so that the robot 100 accelerates and decelerates to accommodate for the velocity of the person 20 and the shutter speed and/or focal limitations of the imaging sensor 450 while positioning itself to capture an image 50 ( e.g., take a
picture) of the moving person 20.)].
Claim 13: wherein the scene attention-getting degree determination unit acquires image analysis information of the moving object and determines the scene attention-getting degree by using the image analysis information in combination with the maneuvering instruction information [0067 (The controller system 500 may execute movement commands to maneuver the robot 100 in relation to the location of the person 20…The movement commands may be based on a trajectory prediction TR and velocity of the person 20, in addition to dynamics of the robot 100 and/or shutter speed of the imaging sensor 450. The controller 500 integrates the movements of the robot 100, the person 20, and the shutter speed and/or focal limitations of the imaging sensor 450 so that the robot 100 accelerates and decelerates to accommodate for the velocity of the person 20 and the shutter speed and/or focal limitations of the imaging sensor 450 while positioning itself to capture an image 50 ( e.g., take a picture) of the moving person 20.)].
Claim 14: wherein the scene attention-getting degree determination unit acquires information on a landmark in the vicinity of the moving object and determines the scene attention-getting degree by using the image analysis information in combination with the information on the landmark [0113 (the robot 100 receives an occupancy map 700 (e.g., from the security system 1000) of objects including walls 18 in a patrolling scene 10 and/or a patrolling area 5…The robot 100 can use the occupancy map 700 to identify and detect people 20 in the scene 10 as well as occlusions 16 (e.g., wherein objects cannot be confirmed from the current vantage point)…The
robot 100 can register an occlusion 16 or wall 18 in the scene 10 and attempt to circumnavigate the occlusion 16 or wall 18 to verify a location of new person 20, 20a-b or other object in the occlusion 16. The robot 100 can register the occlusion 16 or person 20 in the scene 10 and attempt to follow and/or capture a clear still image 50 or video of the person 20.); 0133 (For example, in a facility, such as an office building, the robot 100 may need to know the location of each room, entrances and hallways. The layout map 700 may include fixed obstacles 18, such as
walls, hallways, and/or fixtures and furniture. In some implementations, the robot 100 receives the layout map 700 and can be trained to learn the layout map 700 for autonomous navigation)].
Claim 17: wherein the processor is caused to further function as a predicted panning amount calculation unit configured to calculate a predicted background-panning amount, based on the maneuvering instruction information [0077 (The sensor module 300 may include one or more
imaging sensors 450 of a sensor system 400…that can point or aim at an object that needs more attention. The imaging sensor(s) 450 may reciprocate or scan back and forth slowly as well. The imaging sensor(s) 450 may capture point clouds that are 58 degrees wide and 45 degrees vertical,
at up to 60 Hz.)].
Claim 26: wherein environmental information around the object involving maneuvering is acquired in addition to the maneuvering instruction information [0110 (while patrolling the environment 10, the robot 100 needs to scan the imaging sensor(s) 450 from side to side and/or up and down to detect a person 20 around an occlusion); 0113 (using the occupancy map 700, the robot 100 can determine and track movement of a person 20 in the scene 10. For example, using the imaging sensor 450, the controller 500 may detect movement of the person 20 in the scene 10 and continually update the occupancy map 700 with a current location of the identified person 20.)].
Claim 27: wherein the environmental information is information including at least one of position information of the object involving maneuvering, a weather around the object, a map [0110 (while patrolling the environment 10, the robot 100 needs to scan the imaging sensor(s) 450 from side to side and/or up and down to detect a person 20 around an occlusion); 0113 (using the occupancy map 700, the robot 100 can determine and track movement of a person 20 in the scene 10. For example, using the imaging sensor 450, the controller 500 may detect movement of the person 20 in the scene 10 and continually update the occupancy map 700 with a current location of the identified person 20.)], a structure, sunny/shaded areas, and a time.
Claim 31: wherein in a case where it is difficult to acquire the maneuvering instruction information, the image capturing device controls the image-capturing conditions at the time of image-capturing based on at least image analysis information of the object involving maneuvering [0067 (The controller 500 integrates the movements of the robot 100, the person 20, and the shutter speed and/or focal limitations of the imaging sensor 450 so that the robot 100 accelerates and decelerates to accommodate for the velocity of the person 20 and the shutter speed and/or focal limitations of the imaging sensor 450 while positioning itself to capture an image 50 ( e.g., take a picture) of the moving person 20. The controller 500 predicts the trajectory of the moving person and calculates the stop time and/or deceleration time of the robot 100 and the focal range and shutter speed of the imaging sensor 450 in deciding at which distance from the moving person 20 to capture a photograph or video clip.)].
Claim 32: wherein the image capturing control unit is mounted on the image-capturable moving apparatus, and wherein the acquisition unit acquires the maneuvering instruction information form the moving object by requesting the maneuvering instruction information to the moving object [0065 (a robot 100 patrolling an environment 10 may sense the presence of a person 20 within that environment 10 using one or more sensors, such as…an imagining sensor 450 of a sensor module 300 in communication with a controller system 500 (also referred to as a controller) of the robot 100. The robot 100 may maneuver to have the person 20 within a sensed volume of space S and/or to capture images 50 ( e.g., still images or video) of the person 20 using the imaging sensor 450.)].
Claim 34: wherein a main object is further described in the shooting scenario list [0067 (The controller 500 predicts the trajectory of the moving person and calculates the stop time and/or deceleration time of the robot 100 and the focal range and shutter speed of the imaging sensor 450 in deciding at which distance from the moving person 20 to capture a photograph or video clip. For instance, when the person 20 is running away from the robot 100, the controller system 500 may command the robot 100 to speed up ahead of the person 20 so that the person 20 is centered in the field of view 452 of the imaging sensor 450 once the robot 100 slows, stops and/or catches up to the person 20 for capturing a clear image 50 (i.e. a first choice in the list of appropriate actions). In other situations, the controller 500 may command the robot 100 to back away from the person 20 if the person 20 is determined to be too close to the imaging sensor 450 to capture a crisp image 50 (the second choice in the list of appropriate actions). Moreover, the controller 500 may command the robot 100 to follow the person 20, for example, at a distance, to observe the person 20 for a period of time (a third choice in the list of appropriate actions).)].
Claim Objections
4. In regard to the 35 U.S.C. § 102 rejections noted above, claims 9-11, 15-16, 18-25, 28-30 and 33 are objected to as being dependent upon a rejected base claim.
Prior Art
5. The following prior art, discovered in an updated search and herein made of record, is considered pertinent to Applicant’s disclosure, and consists of documents A-C on the attached PTO-892 Notice of References Cited:
Documents A-C define documents of particular relevance, wherein the claimed invention cannot be considered novel or cannot be considered to involve an inventive step when the document is taken alone.
Prior Art of Record
6. The Examiner has cited particular paragraphs or columns and line numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested of the applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. The prompt development of a clear issue requires that the replies of the Applicant meet the objections to and rejections of the claims. Applicant should also specifically point out the support for any amendments made to the disclosure (see MPEP §2163.06). Applicant is reminded that the Examiner is entitled to give the Broadest Reasonable Interpretation (BRI) of the language of the claims. Furthermore, the Examiner is not limited to Applicant’s definition which is not specifically set forth in the claims. [SEE MPEP 2141.02 [R-07.2015] VI. PRIOR ART MUST BE CONSIDERED IN ITS ENTIRETY, INCLUDING DISCLOSURES THAT TEACH AWAY FROM THE CLAIMS: A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851 (1984). See also MPEP §2123].
In addition, disclosures in a reference must be evaluated for what they would fairly teach one of ordinary skill in the art [See In re Snow, 471 F.2d 1400, 176 USPQ 328 (CCPA 1973) and In re Boe, 355 F.2d 961, 148 USPQ 507 (CCPA 1966)]. Specifically, in considering the teachings of a reference, it is proper to take into account not only the specific teachings of the reference, but also the inferences that one skilled in the art would reasonably have been expected to draw from the reference [See In re Freda, 401 F.2d 825, 159 USPQ 342 (CCPA 1968) and In re Shepard, 319 F.2d 194, 138 USPQ 148 (CCPA 1963)]. Likewise, it is proper to take into consideration not only the teachings of the prior art, but also the level of ordinary skill in the art [See In re Luck, 476 F.2d 650, 177 USPQ 523 (CCPA 1973)]. Specifically, those of ordinary skill in the art are presumed to have some knowledge of the art apart from what is expressly disclosed in the references [See In re Jacoby, 309 F.2d 513, 135 USPQ 317 (CCPA 1962)].
Response Guidelines
7.1 A shortened statutory period for response to this non-final action is set to expire 3 (three) months and 0 (zero) days from the date of this letter. Unless the applicant is notified in writing that a reply is required in less than six months (see the shortened response period previously noted), a maximum period of six months is allowed, if a petition for an extension of time and the fee set in § 1.17(a) are filed [see MPEP 710 and 35 U.S.C. 133]. Failure to respond within the required period for response will cause the application to become abandoned [see MPEP 710.02, 710.02(b)].
7.2 Any response to the Examiner in regard to this non-final action should be
directed to: Russell Frejd, telephone number (571) 272-3779, Monday-Friday from 0730 to
1600 ET. If attempts to reach the examiner by telephone are unsuccessful,
please contact the examiner’s supervisor, Peter Nolan, who can be reached at
(571) 270-7016.
mailed to: Commissioner of Patents and Trademarks
P.O. Box 1450, Alexandria, VA 22313-1450
faxed to: (571) 273-8300
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Hand-delivered responses should be brought to the Customer Service Window, Randolph Building, 401 Dulany Street, Alexandria, VA, 22314.
/RUSSELL FREJD/
Primary Examiner AU 3661
Prosecution Insights