Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Status of Claims
This action is in reply to the application filed on March 25, 2024.
This application claims priority to provisional application 63/460,656, filed April 20, 2023.
Claims 1-20 are currently pending.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 4-6 and 14-16 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the enablement requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to enable one skilled in the art to which it pertains, or with which it is most nearly connected, to make and/or use the invention. Claims 4 and 14 recite “training a foliage detection model that identifies locations where foliage poses a potential future road hazard.” The specification does not detail how one of ordinary skill in the art would train such a model. [0021] states that “there may be a detector trained to detect ruts, potholes, and cracks, a detector trained to detect faded road markings, and a detector trained to detect foliage,” and that “[a] foliage detector may use a depth model to identify the depth of an input image and may convert the resulting depth map to a point cloud.” The specification is silent regarding training a foliage detection model, what type of model is used, and what training data may be used to train the model. Considering the Wands factors:
(A) The breadth of the claims: the claims are broad.
(B) The nature of the invention: the nature of the invention is machine learning.
(C) The state of the prior art: the prior art details training steps and specific models.
(D) The level of one of ordinary skill: the level of ordinary skill is high.
(E) The level of predictability in the art: the level of predictability is low.
(F) The amount of direction provided by the inventor: no direction was provided about training the model.
(G) The existence of working examples: there is no evidence of working examples.
(H) The quantity of experimentation needed to make or use the invention based on the content of the disclosure: a high amount of experimentation would be needed to test various models and different training data.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Independent claims 1 and 11 recite “a remainder of examples from the unlabeled training dataset” but it is not clear what constitutes a “remainder of examples.”
Claims 4-6 and 14-16 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. The term “potential future road hazard” in claims 4 and 14 is a relative term which renders the claim indefinite. The term “potential” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
When considering subject matter eligibility under 35 U.S.C. § 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). The step 2A analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g. mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application. If it is determined that step 2A, Prong that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself.
According to Step 1 of the analysis, in the instant case claims 1-10 are directed to a method and claims 11-20 are directed to a system. Thus, each of the claims falls within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter); claim 18 is not directed to a statutory category, as discussed above, but considered here for compact prosecution.
Considering independent claim 1 and Step 2A, Prong One, the limitations including: “annotating a subset of an unlabeled training dataset, that includes images of road scenes, with labels;” covers performance of the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind.
MPEP 2106.04(a)(2)(III) notes “the "mental processes" abstract idea grouping is defined as concepts performed in the human mind and examples of mental processes include observations, evaluations, judgments, and opinions,” and “including adding pseudo-labels to a remainder of examples from the unlabeled training dataset.” The “annotating” and “adding pseudo-labels” steps in claim 1 are evaluations, judgments, and/or opinions, and mental steps; a human can annotate an image and label images. Therefore, the claim contains abstract elements and the evaluation proceeds to Step 2A, Prong Two.
Considering Step 2A, Prong Two, the judicial exception in claim 1 is not integrated into a practical application. Claim 1 includes the additional elements: “iteratively training a road defect model, … training the road defect detection model based on the labels and the pseudo-labels.” The training is not detailed and is a tangential addition to the claim; see MPEP 2106.05(g).
Considering Step 2B, the additional elements do not amount to significantly more. The training is insignificant post-solution activity, as the inference is tangential to the analysis and selection of classification; see MPEP 2106.05(g).
Therefore, claim 1 is ineligible in view of 35 U.S.C. 101.
Claims 2-4, dependent on claim 1, do not contain abstract ideas. Claims 2-4 include only additional elements that would be analyzed under Step 2A, Prong Two, and Step 2B, if the claim is incorporated into claim 1. Claims 2-4 are rejected due to their dependence on rejected claim 1.
Considering claim 5, dependent on claim 4, and Step 2A, Prong One, the limitations including: “the foliage detection model identifies pixels that correspond to foliage and pixels that correspond to a road in an input image and determines locations where the foliage extends over the road” covers performance of the mind. MPEP 2106.04(a)(2)(III) notes “the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions.” Claim 5’s identification and determination is an evaluation, judgment, and/or opinion.
Claim 5 does not contain any new additional elements and, therefore, is not integrated into a practical application or amount to significantly more under Step 2A, Prong Two, and Step 2B.
Therefore, claim 5 is ineligible in view of 35 U.S.C. 101.
Claim 6, dependent on claim 5, does not contain abstract ideas. Claim 6 includes only additional elements that would be analyzed under Step 2A, Prong Two, and Step 2B, if the claim is incorporated into claims 1 or 5. Claim 6 is rejected due to their dependence on rejected claims 1 and 5.
Claims 7-9, dependent on claim 1, do not contain abstract ideas. Claims 7-9 include only additional elements that would be analyzed under Step 2A, Prong Two, and Step 2B, if the claims are incorporated into claim 1. Claims 7-9 are rejected due to their dependence on rejected claim 1.
Considering claim 10, dependent on claim 1, and Step 2A, Prong One, the limitations including: “identifying a defect of a road in the road scene” covers performance of the mind. MPEP 2106.04(a)(2)(III) notes “the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions.” Claim 10’s identification is an evaluation, judgment, and/or opinion.
Considering Step 2A, Prong Two, the judicial exception in claim 10 is not integrated into a practical application. Claim 10 includes the additional elements “capturing a new image of a road scene” and “automatically operating a vehicle to avoid the defect.” The capturing is insignificant extra-solution activity, mere data gathering; see MPEP 2106.05(g). Further, the training is not detailed and is a tangential addition to the claim; see MPEP 2106.05(g).
Considering Step 2B, the additional elements do not amount to significantly more. The capturing and training are insignificant post-solution activity; see MPEP 2106.05(g).
Therefore, claim 1 is ineligible in view of 35 U.S.C. 101.
Independent claim 11 is similar to claim 1 and rejected for the same reasons as disclosed above. Claim 11 includes additional elements, including a hardware processor and memory, but these elements do not integrate the abstract idea into a practical application or amount to significantly more. The elements are generic computer components; see MPEP 2106.05(f).
Claims 12-20 are similar to claims 2-10 and rejected for the same reasons as disclosed above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 2, 7-12, and 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sunde et al., U.S. Patent Application Publication 2022/0101272 (“Sunde”), in view of Leng et al., U.S. Patent Application Publication 2022/0156585 (“Leng”).
With respect to independent claim 1, Sunde teaches:
A computer-implemented method for training a model, comprising:
annotating a subset of an unlabeled training dataset, that includes images of road scenes, with labels (Sunde teaches collecting video images of roadway condition indicia and a machine learning algorithm that labels each video image with an initial indicator of an area or item of concern; see claim 1.); and
iteratively training a road defect detection model (Sunde teaches training a machine learning algorithm to identify items, including cracking, patches, potholes, in [0044]-[0045].),
Sunde does not explicitly disclose:
including adding pseudo-labels to a remainder of examples from the unlabeled training dataset and training the road defect detection model based on the labels and the pseudo-labels.
However, Leng teaches these features:
including adding pseudo-labels to a remainder of examples from the unlabeled training dataset and training the road defect detection model based on the labels and the pseudo-labels (Leng teaches generating pseudo-labels in an autonomous vehicle in [0034]-[0035].).
Sunde and Leng are analogous art directed detection using machine learning models. Sunde teaches training a model to detect various road features and Leng teaches using pseudo-labels to train a system to detect objects in an automotive environment.
It would have been obvious for one of ordinary skill in neural networking to implement Leng’s pseudo-labeling methods into Sunde’s disclosed system before the filing date of the claimed invention. It would have been obvious because one of ordinary skill would be motivated to increase flexibility and improve efficiency as detailed in [0017] of Leng.
With respect to independent claim 11, Sunde teaches implementation including a hardware processor and memory; see [0022]. The remaining features in claim 11 are similar to claim 1 and rejected for the same reasons as presented above.
With respect to dependent claims 2 and 12, the rejections of claims 1 and 11 are incorporated. Further, Sunde teaches:
wherein the training dataset includes images that depict multiple categories of road defect, including cracks, ruts, and faded road markings (Sunde teaches images may be analyzed by a machine-learning algorithm to determine cracks and transverse cracks (ruts) in [0045]. [0053] also teaches assessing factors including pavement markings.).
With respect to dependent claims 7 and 17, the rejections of claims 1 and 11 are incorporated. Further, Leng teaches:
wherein iteratively training the road defect detection model includes adding the pseudo-labels to examples from the unlabeled training dataset, the pseudo-labels having a confidence value that is higher than a threshold value, to be used in a next iteration of the training (Leng teaches generating pseudo-labels in an autonomous vehicle and training over multiple generations in [0034]-[0036]. Leng further teaches determining a confidence score threshold for each hyperparameter in [0040].).
See the rejection of claim 1 for the motivation to combine references.
With respect to dependent claims 8 and 18, the rejections of claims 7 and 17 are incorporated. Further, Leng teaches:
wherein the iterative training terminates after all of the remainder of examples from the unlabeled training dataset have pseudo-labels having a confidence value that is higher than the threshold value (Leng teaches selecting training candidates having a best measure of performance and using those candidates to train a system in [0044]-[0049].).
See the rejection of claim 1 for the motivation to combine references.
With respect to dependent claims 9 and 19, the rejections of claims 7 and 17 are incorporated. Further, Leng teaches:
wherein the iterative training terminates after a predetermined number of iterations, with any examples of the remainder of examples from the unlabeled training dataset that do not have pseudo-labels having a confidence value that is higher than the threshold value being omitted from training (Leng teaches selecting training candidates having a best measure of performance and using those candidates to train a system in [0044]-[0049]. Leng further teaches stopping iteration in [0041].).
See the rejection of claim 1 for the motivation to combine references.
With respect to dependent claims 10 and 20, the rejections of claims 1 and 11 are incorporated. Further:
capturing a new image of a road scene (Sunde teaches capturing data in figure 1 and [0065].);
identifying a defect of a road in the road scene (Sunde teaches captured images may be analyzed to detect road distresses, including cracking, patches, and potholes in [0045].); and
Sunde does not explicitly disclose, but Leng teaches:
automatically operating a vehicle to avoid the defect (Leng teaches robotic control and autonomous vehicle motion planning in [0015].).
See the rejection of claim 1 for the motivation to combine references.
Claim(s) 3-6 and 13-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sunde et al., U.S. Patent Application Publication 2022/0101272 (“Sunde”), in view of Leng et al., U.S. Patent Application Publication 2022/0156585 (“Leng”); and in further view of Huang et al., “Traffic Sign Occlusion Detection Using Mobile Laser Scanning Point Clouds” (“Leng”).
With respect to dependent claims 3 and 13, the rejections of claims 1 and 11 are incorporated. Further Sunde and Leng do not teach:
wherein the training dataset further includes images that depict foliage.
However, Huang teaches this feature:
Huang teaches images that depict foliage in at least figure 3.
Sunde, Leng, and Huang are analogous art directed detection using machine learning models. Sunde teaches training a model to detect various road features, Leng teaches using pseudo-labels to train a system to detect objects in an automotive environment, and Huang teaches a model for detecting traffic sign occlusion.
It would have been obvious for one of ordinary skill in neural networking to implement Haung’s occlusion detection methods into Sunde’s disclosed system before the filing date of the claimed invention. It would have been obvious because one of ordinary skill would be motivated to provide a maintenance guide for traffic facility maintainer and planners, as detailed in section I of Huang.
With respect to dependent claims 4 and 14, the rejections of claims 3 and 13 are incorporated. Further, Huang teaches:
training a foliage detection model that identifies locations where foliage poses a potential future road hazard (Huang teaches that sign occlusion may worsen with plant growth in the second full paragraph on page 2369.).
See the rejection of claim 3 for the motivation to combine references.
With respect to dependent claims 5 and 15, the rejections of claims 4 and 14 are incorporated. Further, Huang teaches:
wherein the foliage detection model identifies pixels that correspond to foliage and pixels that correspond to a road in an input image and determines locations where the foliage extends over the road (Figure 3 of Huang removes ground pixels (i.e. detects road pixels) and separates foliage.).
See the rejection of claim 3 for the motivation to combine references.
With respect to dependent claims 6 and 16, the rejections of claims 5 and 15 are incorporated. Further, Huang teaches:
wherein the foliage detection model combines semantic segmentation of the input image (Figure 3 of Huang removes ground pixels (i.e. detects road pixels) and separates foliage. Segmenting the road from the rest of the image is a semantic segmentation.) with a depth map based on the input image to identify overlapping foliage points and road points (Huang teaches determining sign occlusion in section III.C., including foliage overlapping a sign.).
See the rejection of claim 3 for the motivation to combine references.
Prior Art of Record
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Caine et al., Patent Application Publication 2022/0180193 – teaches object detection using pseudo-labels.
Pang et al., Patent Application Publication 2023/0282000 – teaches a model for object tracking in an automotive context.
Zhou et al., Patent Application Publication 2023/0055334 – teaches a deep learning model to determine road conditions.
Kim et al., “Learning Semantic Segmentation from Multiple Datasets with Label Shifts” – teaches semantic segmentation details.
Konig et al., “Weakly-Supervised Surface Crack Segmentation by Generating Pseudo-Labels using Localization with Classifier and Thresholding” – teaches detecting cracks in a roadway using pseudo-labels.
Conclusion
Claims 1-20 are rejected.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL T PELLETT whose telephone number is (571)270-7156. The examiner can normally be reached on Monday - Friday 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached on 571-272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL T PELLETT/Primary Examiner, Art Unit 2121
Prosecution Insights