DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged that application claims priority to foreign application with application number JP2023-178241 dated 10/16/2023. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The IDS dated 9/25/2024, 5/14/2025, and 4/2/2026 has been considered and placed in the application file.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f), is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“a change information acquisition unit that acquires change information regarding a change in an appearance state of a target region” in claim 1;
“a training unit that trains a machine learning model using a plurality of sets of training data in which a training image including the target region after a change in the appearance state and the moving object is associated with a correct answer label” in claim 1, 5, and 9;
“And a notification control unit that notifies an administrator of training request information for training the machine learning model” in claim 1.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim(s) 4, 8, and 12 are rejected under 35 U.S.C. 112(b), as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim 4 (and correspondingly claim 8 and 12) recite “wherein the training image includes at least one of the captured image and a composite image generated by combining a virtual moving object image generated by virtually reproducing the moving object with a background image that is either a real background image acquired by the external camera imaging the target region or a virtual background image generated by virtually reproducing the target region”. It is unclear what the Applicant is claiming when stating a composite image is generated by combining a virtual moving object image generated by virtual reproducing the moving object with a background. The examiner recommends amending to clarify the language and what the Applicant is intending to claim. For examination purposes, the examiner will interpret this limitation as wherein the training images include a virtual moving object with a background that is either a real background captured by a camera or a virtual background representing the target region.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4 are rejected under 35 U.S.C. 103 as being unpatentable over Jiao (US 20210027622 A1) in view of Schulter (US 20230281999 A1), Shikanai (US 20220412756 A1), and Ponnalagu (US 20250077949 A1).
Regarding claim 1, Jiao discloses an information processing device used to move a moving object that is capable of moving by unmanned driving (Jiao, paragraph [0023], "In one embodiment, the vehicle 102 is driven autonomously") comprising:
a change information acquisition unit that acquires change information regarding a change in an appearance state of a target region (Jiao, paragraph [0049], "At step 302, one or more sensors of the vehicle 102 detect a road condition”, a road condition includes potholes, debris, etc. i.e. change from a normal road/pathway).
While Jiao teaches including a pathway on which the moving object moves and a surrounding area of the pathway (Jiao, paragraph [0049], "In an exemplary embodiment, the road condition component 112 analyzes data from the external sensors 122 for a single type of road condition such as a fire."), they do not teach “the target region being included in an imaging range of an external camera provided in a place different from a place of the moving object”.
However, Schulter teaches the target region being included in an imaging range of an external camera provided in a place different from a place of the moving object (Schulter, paragraph [0067], "The scene 900 is monitored by one or more video cameras 908, which may be positioned on utility poles as illustrated to obtain an overhead perspective of the scene 900").
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to capture images of Jiao’s target region and vehicle from an overhead camera, as taught by Schulter.
The suggestion/motivation for doing so would have been to encompass a broader region of detection, enhancing awareness and security.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Jiao in view of Schulter discloses and when the appearance state is determined, using the change information, to have changed, at least one of: a training unit that trains a machine learning model using a plurality of sets of training data in which a training image including the target region after a change in the appearance state and the moving object is associated with a correct answer label (Jiao, paragraph [0041], "In one embodiment, the pothole component 205 is trained by a machine learning algorithm to identify potholes from camera 130 images by observing a multitude of images of potholes", the vehicle is now included in the images as modified above) when a captured image acquired by the external camera imaging the target region and the moving object is input (Schulter, paragraph [0067], "The scene 900 is monitored by one or more video cameras 908, which may be positioned on utility poles as illustrated to obtain an overhead perspective of the scene 900").
While Jiao in view of Schulter discloses defining operation of the moving object that moves by the unmanned driving and a parameter used in generating the control signal (Jiao, paragraph [0056], "At step 310, the road condition information is transmitted such that the road condition information is receivable by the subset of other vehicles 150", determining other vehicles being affected by the road condition is the parameter and transmitting this info is an operation of a vehicle), they do not do so through a machine learning model.
However, Shikanai teaches the machine learning model being configured to output at least one of a control signal for defining operation of the moving object that moves by the unmanned driving and a parameter used in generating the control signal (Shikanai, paragraph [0009], "The estimation unit is configured to estimate the road surface state information by inputting the vehicle state information acquired by the acquisition unit to a trained model that outputs the road surface state information in a case where the vehicle state information is input and that has been trained in advance based on training data in which the vehicle state information and the road surface state information are associated with each other").
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use a machine learning model to transmit Jiao’s (in view of Schulter) information, as taught by Shikanai.
The suggestion/motivation for doing so would have been to automate the process of transmission.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Jiao in view of Schulter and Shikanai does not teach “And a notification control unit that notifies an administrator of training request information for training the machine learning model”.
However, Ponnalagu teaches a notification control unit that notifies an administrator of training request information for training the machine learning model (Ponnalagu, paragraph [0028], " For example, the training service 112 can be executed to determine a frequency for retraining the machine learning model 154, determine whether to replace a deployed machine learning model 154 with a challenger model for a deployment environment, notify model users of retraining characteristics 131 for a particular machine learning model 154, and other suitable aspects.").
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to notify a user for training Jiao’s (in view of Schulter, Shikanai, and Ponnalagu) model, as taught by Ponnalagu.
The suggestion/motivation for doing so would have been to raise awareness and allow users to confirm the parameters of the machine learning.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Jiao in view of Schulter, Shikanai, and Ponnalagu to obtain the invention as specified in claim 1.
Regarding claim 2, Jiao in view of Schulter, Shikanai, and Ponnalagu discloses the information processing device according to claim 1, wherein a factor affecting a change in the appearance state includes at least one of dirt in the target region, irregularities in the target region, an object other than the moving object and movably disposed in the target region, and a road surface color of the pathway (Jiao, paragraph [0064], Fig. 6 below, "Referring to FIG. 6, FIG. 6 is an illustration 600 of a multitude of road conditions that are lane specific on a road. The road condition component 112 may be configured to identify many types of different road conditions at once", dirt pertains to potholes, irregularities are any abnormal road conditions, objects can be deer, and potholes are another form of a road surface color of a pathway).
PNG
media_image1.png
764
507
media_image1.png
Greyscale
Regarding claim 3, Jiao in view of Schulter, Shikanai, and Ponnalagu discloses the information processing device according to claim 1, wherein the machine learning model is trained during a period when the unmanned driving control is performed (Jiao, paragraph [0043], "To identify debris from camera 130 image data, the debris component 215 may be trained by a machine learning algorithm to identify debris from camera 130 images by observing multiple images of various types of debris and matching the currently observed debris with previously identified debris", the machine learning algorithm inputs images taken during the vehicle drive, thus they are trained while driving).
Regarding claim 4, Jiao in view of Schulter, Shikanai, and Ponnalagu discloses the information processing device according to claim 1.
Jiao in view of Schulter, Shikanai, and Ponnalagu does not teach “wherein the training image includes at least one of the captured image and a composite image generated by combining a virtual moving object image generated by virtually reproducing the moving object with a background image that is either a real background image acquired by the external camera imaging the target region or a virtual background image generated by virtually reproducing the target region”.
However, Schutler additionally teaches wherein the training image includes at least one of the captured image and a composite image generated by combining a virtual moving object image generated by virtually reproducing the moving object with a background image that is either a real background image acquired by the external camera imaging the target region or a virtual background image generated by virtually reproducing the target region (Schulter, paragraph [0070], Fig. 9 below, "During image segmentation 310, a new image of a road scene 900 is acquired at block 1012, for example being generated by a camera 908 that overlooks the scene 900. Block 1014 then performs panoptic image segmentation using a trained model to identify masks within the image that correspond to different objects", panoptic segmentation results in a virtual vehicle with a virtual background).
PNG
media_image2.png
602
512
media_image2.png
Greyscale
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use panoptic images as training images for Jiao’s (in view of Schulter, Shikanai, and Ponnalagu) model, as additionally taught by Schulter.
The suggestion/motivation for doing so would have been to provide more detailed and labelled images for more accurate model predictions.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Jiao in view of Schulter, Shikanai, Ponnalagu and with the additional teachings of Schulter to obtain the invention as specified in claim 4.
Claim(s) 5-12 are rejected under 35 U.S.C. 103 as being unpatentable over Jiao (US 20210027622 A1) in view of Schulter (US 20230281999 A1) and Shikanai (US 20220412756 A1).
Regarding claim 5, Jiao discloses an information processing device used to move a moving object that is capable of moving by unmanned driving (Jiao, paragraph [0023], "In one embodiment, the vehicle 102 is driven autonomously") comprising:
a training unit that trains a machine learning model when a captured image (Jiao, paragraph [0041], "In one embodiment, the pothole component 205 is trained by a machine learning algorithm to identify potholes from camera 130 images by observing a multitude of images of potholes").
Jiao does not teach “the captured image acquired by an external camera imaging a target region and the moving object is input”.
However, Schulter teaches the captured image acquired by an external camera imaging a target region and the moving object is input (Schulter, paragraph [0067], "The scene 900 is monitored by one or more video cameras 908, which may be positioned on utility poles as illustrated to obtain an overhead perspective of the scene 900").
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to capture images of Jiao’s target region and vehicle from an overhead camera, as taught by Schulter.
The suggestion/motivation for doing so would have been to encompass a broader region of detection, enhancing awareness and security.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
While Jiao in view of Schulter teaches discloses defining operation of the moving object that moves by the unmanned driving and a parameter used in generating the control signal (Jiao, paragraph [0056], "At step 310, the road condition information is transmitted such that the road condition information is receivable by the subset of other vehicles 150"), they do not do so through a machine learning model.
However, Shikanai teaches the machine learning model being configured to output at least one of a control signal for defining operation of the moving object that moves by the unmanned driving and a parameter used in generating the control signal (Shikanai, paragraph [0009], "The estimation unit is configured to estimate the road surface state information by inputting the vehicle state information acquired by the acquisition unit to a trained model that outputs the road surface state information in a case where the vehicle state information is input and that has been trained in advance based on training data in which the vehicle state information and the road surface state information are associated with each other").
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use a machine learning model to transmit Jiao’s (in view of Schulter) information, as taught by Shikanai.
The suggestion/motivation for doing so would have been to automate the process of transmission.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Jiao in view of Schulter and Shikanai discloses the target region being included in an imaging range of the external camera provided in a place different from a place of the moving object (Schulter, paragraph [0067], "The scene 900 is monitored by one or more video cameras 908, which may be positioned on utility poles as illustrated to obtain an overhead perspective of the scene 900"), and including a pathway on which the moving object moves and a surrounding area of the pathway (Jiao, paragraph [0049], "In an exemplary embodiment, the road condition component 112 analyzes data from the external sensors 122 for a single type of road condition such as a fire."),
wherein the target region is classified into a plurality of appearance states according to a change in appearance (Jiao, paragraph [0049], "At step 302, one or more sensors of the vehicle 102 detect a road condition"),
and the training unit trains the machine learning model for each of the appearance states using M (M is an integer of 2 or more) sets of training data in which a training image including the target region classified as one of the plurality of appearance states and the moving object is associated with a correct answer label (Jiao, paragraph [0041], "In one embodiment, the pothole component 205 is trained by a machine learning algorithm to identify potholes from camera 130 images by observing a multitude of images of potholes"), and N (N is an integer of 0 or more and less than M) sets of training data in which a training image including the target region classified as another remaining appearance state and the moving object is associated with a correct answer label (Jiao, paragraph [0042], “To identify decaying infrastructure from camera 130 image data, the decaying infrastructure component 210 may be trained by a machine learning algorithm to identify decaying infrastructure from camera 130 images by observing a multitude of images of decaying infrastructure”).
Therefore, it would have been obvious to combine Jiao in view of Schulter and Shikanai to obtain the invention as specified in claim 5.
Claims 6-8 corresponds to claims 2-4 respectively. Thus, they are rejected for the same reasons of obviousness as claims 2-4.
Claim 7 corresponds to claim 5. Thus, they are rejected for the same reasons of obviousness as claim 5.
Claims 10-12 corresponds to claims 2-4. Thus, they are rejected for the same reasons of obviousness as claims 2-4.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WAYNE ZHANG whose telephone number is (571) 272-0245. The examiner can normally be reached Monday-Friday 10:00-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Sumati Lefkowitz can be reached on (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WAYNE ZHANG/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights