Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
In January, 2019 (updated October 2019), the USPTO released new examination guidelines setting forth a two-step inquiry for determining whether a claim is directed to non-statutory subject matter. According to the guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that claim 1 is directed toward non-statutory subject matter, as shown below:
STEP 1: Does claim 1 fall within one of the statutory categories? Yes. The claim is directed toward a process, which falls within one of the statutory categories.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea? Yes, the claim is directed to an abstract idea.
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
Claim 1 recites:
A computer-implemented method for sensor fusion in relation to at least partially autonomous driving of a vehicle, the method comprising:
obtaining first signatures of first patches of a first type sensed information unit (SIU) that was sensed by a first sensor of a first type;
obtaining second signatures of second patches of a second type SIU that was sensed by a second sensor of a second type, the second type differs from the first type; wherein the first sensor and the second sensor are associated with the vehicle;
finding correlations by applying a correlation function between the first signatures and the second signatures; wherein the finding is executed by a mapping system; and
determining, based on the correlations and by the mapping system, a mapping between the first patches and the second patches, the mapping to be used in an at least partially autonomous driving of a vehicle;
wherein the correlation function having been developed by applying a supervised machine learning process based on relationships between members of training signature pairs, each training signature pair comprises a first sensor training signature and a second sensor training signature of a same object.
The highlighted portion of claim 1 above is a mental process that can be practicably performed in the human mind and, therefore, an abstract idea. It merely consists of finding correlations between signatures of patches and determining a mapping between the patches based on the correlation. This is equivalent to a person looking at similar regions (e.g. a common object) of two images of an object from different perspectives and determining the positional relationship between the perspectives based on that. The Examiner notes that under MPEP 2106.04(a)(2)(III), the courts consider a mental process (thinking) that "can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 ("‘[M]ental processes[] and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978) (same). As such, a person observing two images could deduce the shift in perspective from one to the other. The mere nominal recitation that the process is being executed by a computer does not take the limitation out of the mental process grouping. Notably, the claim does not positively recite any limitations regarding actual use of the mapping in a specific manner to fuse the sensor data, nor does the claim recite using the mapping to aid in controlling the autonomous vehicle in a specific manner.
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application? No, the claim does not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Claim 1 recites:
A computer-implemented method for sensor fusion in relation to at least partially autonomous driving of a vehicle, the method comprising:
obtaining first signatures of first patches of a first type sensed information unit (SIU) that was sensed by a first sensor of a first type;
obtaining second signatures of second patches of a second type SIU that was sensed by a second sensor of a second type, the second type differs from the first type; wherein the first sensor and the second sensor are associated with the vehicle;
finding correlations by applying a correlation function between the first signatures and the second signatures; wherein the finding is executed by a mapping system; and
determining, based on the correlations and by the mapping system, a mapping between the first patches and the second patches, the mapping to be used in an at least partially autonomous driving of a vehicle;
wherein the correlation function having been developed by applying a supervised machine learning process based on relationships between members of training signature pairs, each training signature pair comprises a first sensor training signature and a second sensor training signature of a same object.
The highlighted portion of claim 1 above does not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application. While the claim does recite that the method is “for sensor fusion in relation to at least partially autonomous driving of a vehicle”, there are no limitations in the body of the claim that recite determining the actual attitude. There is no hint of the vehicle or its computer being a particular machine or manufacture that is integral to the claim, either. The computer merely receives the data and performs the mental process without performing any further functions. The data is not transformed by any steps of the method, or used implement a specific control of the vehicle.
Also, as noted above, merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea is indicative that the judicial exception has not been integrated into a practical application. In the instant case, the steps are “computer-implemented” and thus the claim merely applies the mental process to a generic computer environment, which is indicative of the abstract idea having not been integrated into a practical application. Further, the machine learning process is only generically “appl[ied]” to the correlation function, which does not rise to the level of integration into practical application.
The obtaining steps recited in the claim are recited at a high level of generality (i.e., as a general means of gathering an electronic representation of an area or navigational data or planned path data), and amount to mere data gathering, which is a form of insignificant extra-solution activity.
The one or more data networks, one or more processors, one or more memories storing computer readable instructions, and the computer readable storage medium comprising computer-readable instructions merely describes how to generally “apply” the otherwise mental judgments in a generic or general-purpose computing environment. The one or more data networks, one or more processors, one or more memories storing computer readable instructions, and the computer readable storage medium comprising computer-readable instructions are recited at a high level of generality and merely automate the generating steps.
The additional limitation of a sensor is claimed generically and are operating in their ordinary capacity such that they do not use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more is more than a drafting effort designed to monopolize the exception.
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No, the claim does not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
Claim 1 does not recite any specific limitation or combination of limitations that are not well-understood, routine, conventional (WURC) activity in the field. Obtaining and applying functions to data are both fundamental, i.e. WURC, activities performed by computers, such as the computer in claim 1. Further, applicant’s specification does not provide any indication that the method steps are performed using anything other than a conventional computer. MPEP 2106.05(d)(II), and the cases cited therein, including Intellectual Ventures I, LLC v. Symantec Corp., 838 F.3d 1307, 1321 (Fed. Cir. 2016), TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610 (Fed. Cir. 2016), and OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015), indicate that mere performance of an action is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner (as it is here).
CONCLUSION
Thus, since claim 1 is: (a) directed toward an abstract idea, (b) does not recite additional elements that integrate the judicial exception into a practical application, and (c) does not recite additional elements that amount to significantly more than the judicial exception, it is clear that claim 1 is directed towards non-statutory subject matter.
Independent claim 9 has similar limitations to claim 1 above, and is therefore rejected based on a similar rationale.
Dependent claims 2-8 are likewise rejected as ineligible. They either add to the mental process (claims 2-4 and 6-7) or add post solution activity (claims 5 and 8). Specifically, claim 5 merely adds a nominal sensor fusion that merely applies the mental process at a high level, while claim 8 simply adds generic data manipulation associated with a “fusing” without going beyond a very high level in the fusing. Thus, the dependent claims are likewise ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9 are rejected under 35 U.S.C. 103 as being unpatentable over US20230009766 by Ferroni (hereinafter “Ferroni”), further in view of US12409859 by Burlina et al. (hereinafter “Burlina”).
Regarding claim 1, Ferroni teaches A computer-implemented method for sensor fusion in relation to at least partially autonomous driving of a vehicle, see for example paragraph [0058] describing the application of the method to an autonomous vehicle.
the method comprising: obtaining first signatures of first patches of a first type sensed information unit (SIU) that was sensed by a first sensor of a first type; obtaining second signatures of second patches of a second see for example paragraphs [0055]-[0062] and [0038], where an autonomous vehicle receives sensor data from multiple sensors of an object in a cell, where each sensor sees features of a recognizable object (signature) in an area (patch) of an image plane for each sensor.
finding correlations by applying a correlation function between the first signatures and the second signatures; wherein the finding is executed by a mapping system; see for example paragraphs [0056]-[0058], where the system determines a correlation between regions sensed in each sensor based on object feature data.
and determining, based on the correlations and by the mapping system, a mapping between the first patches and the second patches, the mapping to be used in an at least partially autonomous driving of a vehicle; see for example paragraphs [0060]-[0065], where the system determines a mapping between sensors based on the derived correlations.
wherein the correlation function having been developed by applying a supervised machine learning process based on relationships between members of training signature pairs, each training signature pair comprises a first sensor training signature and a second sensor training signature of a same object. See for example paragraph [0063], where the data fusion is based on sensor data where the correct output data are known.
Ferroni does not explicitly teach that the second patch is obtained by a second sensor of a second type, the second type differs from the first type. Although Ferroni strongly suggests as much in, e.g., [0065], Ferroni does not explicitly teach correlating a second sensor of a different type to a first type.
However, Burlina teaches that the second patch is obtained by a second sensor of a second type, the second type differs from the first type. See for example col. 3 ll. 16-31, where the system aligns different sensor data from different types of sensors, such as an image sensor and infrared sensor. See also, e.g., col. 2 ll. 27-61 describing more broadly combining different sensor modalities, such as image, lidar, radar, infrared, etc.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the sensor fusion system of Ferroni with the multi-modal sensor fusion of Burlina with a reasonable expectation of success. Doing so allows the system to better detect objects with different sensor modalities that have advantages in different situations, improving the safety of the vehicle.
Claim 9 has similar limitations to claim 1 above, and is therefore rejected based on a similar rationale.
Regarding claim 2, Ferroni does not explicitly teach, but Burlina teaches wherein the determining of the mapping comprises determining a projective transformation. See for example col. 3 ll. 16-31, where the system aligns different sensor data from different types of sensors, such as an image sensor and infrared sensor, using a homography.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the sensor fusion system of Ferroni with the multi-modal sensor fusion of Burlina with a reasonable expectation of success. Doing so allows the system to better detect objects with different sensor modalities that have advantages in different situations, improving the safety of the vehicle.
Regarding claim 3, Ferroni teaches wherein the determining of the mapping further comprises determining a profile of a road on which the vehicle propagates. See for example paragraph [0058], where the vehicle recognizes the road it is traveling on.
Regarding claim 4, Ferroni teaches wherein the determining of the mapping further comprises determining a first sensor orientation parameter. See for example paragraph [0020], where the calculation is based in part on extrinsic sensor parameters.
Regarding claim 5, Ferroni teaches further comprising performing the sensor fusion. See paragraphs [0060]-[0063] where the sensor fusion is performed.
Regarding claim 6, Ferroni teaches further comprising selecting the first patches and the second patches, the first patches are selected from first patches candidates, the second patches are selected from second patches candidates. See for example paragraphs [0056]-[0058], where the system determines a correlation between regions sensed in each sensor based on object feature data.
Regarding claim 7, Ferroni teaches wherein the selecting is based on an estimated mapping between the first patches and the second patches. See again paragraph [0020] describing sensor intrinsic and extrinsic parameters, and paragraph [0035] describing placing the sensor data into a world coordinate system.
Regarding claim 8, Ferroni teaches further comprising fusing content associated with pairs of patches, each pair comprises a first patch and a corresponding second patch that is mapped, according to the mapping, to the first patch. See for example paragraphs [0056]-[0058], where the system determines a correlation between regions sensed in each sensor based on object feature data. See also paragraphs [0060]-[0063] where the sensor fusion is performed.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US12525025 by Burlina et al. teaching homography matching of IR sensor data and image data based on keypoints.
US20210134079 by Nee teaching multimodal sensor calibration (e.g. IR and camera) based on projective transformations (homography).
US10678260 by Mou teaching multimodal sensor calibration by identifying edge regions in order to derive coordinate transformations.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JORDAN THOMAS SMITH whose telephone number is (571)272-0522. The examiner can normally be reached Monday - Friday, 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Antonucci can be reached at (313) 446-6519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JORDAN T SMITH/ Examiner, Art Unit 3666
Prosecution Insights