Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The United States Patent & Trademark Office appreciates the application that is submitted by the inventor/assignee. The United States Patent & Trademark Office reviewed the following application and has made the following comments below.
Priority
This application claims benefit of foreign priority under 35 U.S.C. 119(a)-(d) of 10-2024-0015398, filed in Republic of Korea on 1/31/2024.
Claim Objections
Claims 1-9 are objected to because of the following informalities: “AVM” is not defined in the claims and not sure what that means without looking in specification. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. When reviewing independent claim 1, and based upon consideration of all of the relevant factors with respect to the claim as a whole, claim(s) 1-9 are held to claim an abstract idea without reciting elements that amount to significantly more than the abstract idea and is/are therefore rejected as ineligible subject matter under 35 U.S.C. 101. The Examiner will analyze Claim 1, and similar rationale applies to independent Claim/s 8 and 9. The rationale, under MPEP § 2106, for this finding is explained below:
The claimed invention (1) must be directed to one of the four statutory categories, and (2) must not be wholly directed to subject matter encompassing a judicially recognized exception, as defined below. The following two step analysis is used to evaluate these criteria.
Step 1: Is the claim directed to one of the four patent-eligible subject matter categories: process, machine, manufacture, or composition of matter?
When examining the claim under 35 U.S.C. 101, the Examiner interprets that the claims is related to a process since the claim is directed to an AVM calibration method by using generative artificial intelligence.
Step 2a, Prong 1: Does the claim wholly embrace a judicially recognized exception, which includes laws of nature, physical phenomena, and abstract ideas, or is it a particular practical application of a judicial exception?
The Examiner interprets that the judicial exception applies since Claim 1 limitation of acquiring camera image, initializing camera parameters, performing full-automatic mode AVM, and performing full automatic mode AVM calibration, performing, by a policy network, adjustment, performing, by a value network, secondary filtering, performing, by a control network, additional adjustment are directed to an abstract idea. The claim is related to mathematical relationship by organizing information and manipulating information through mathematical correlations, Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014). The patentee in Digitech claimed methods of generating first and second data by taking existing information, manipulating the data using mathematical functions, and organizing this information into a new form. The court explained that such claims were directed to an abstract idea because they described a process of organizing information through mathematical correlations, like Flook's method of calculating using a mathematical formula. 758 F.3d at 1350, 111 USPQ2d at 1721.
If the claim recites a judicial exception (i.e., an abstract idea enumerated in MPEP § 2106.04(a), a law of nature, or a natural phenomenon), the claim requires further analysis in Prong Two.
Step 2a, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
The Examiner interprets that Claim 1 limitation does not provide additional elements or combination of additional elements to a practical application since the claim/s is generally linking the use of the judicial exception to a particular technological environment or field of use – see MPEP 2106.05(h). See, MPEP §2106.04(d), In short, first the specification should be evaluated to determine if the disclosure provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. The specification need not explicitly set forth the improvement, but it must describe the invention such that the improvement would be apparent to one of ordinary skill in the art. Conversely, if the specification explicitly sets forth an improvement but in a conclusory manner (i.e., a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology. Here, the Examiner interprets that specification does not provide the improvement and does not explicitly set forth an improvement but must make conclusory manner, (i.e., a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art). Thus, the Exmainer finds there is no judicial exception.
Step 2b: If a judicial exception into a practical application is not recited in the claim, the Examiner must interpret if the claim recites additional elements that amount to significantly more than the judicial exception.
The Examiner interprets that the Claims do not amount to significantly more since the Claim/s is/state
ii. Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, e.g., a claim to an abstract idea requiring no more than a generic computer to perform generic computer functions that are well-understood, routine and conventional activities previously known to the industry, as discussed in Alice Corp., 573 U.S. at 225, 110 USPQ2d at 1984 (see MPEP § 2106.05(d));
Furthermore, the generic computer components of the computer recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system.
Claims 2-7 depending on the independent claim/s include all the limitation of the independent claim. The Examiner finds that Claim 2-7 does not states significantly more since the claim only recites they add more mathematical relationship between the claims.
Thus, Claims 1-9 recite the same abstract idea and therefore are not drawn to the eligible subject matter as they are directed to the abstract idea without significantly more.
Therefore, the Examiner interprets that the claims are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 & 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over Heidi et al (U.S. Patent Pub. No. 2016/0343136, hereafter referred to as Heidi) in view of Chakravarty et al (U.S. Patent Pub. 2023/0097584, hereafter referred to as Chakravarty) in view of Ren et al (U.S. Patent Pub. No. 2024/0104941, hereafter referred to as Ren).
Regarding Claim 1, Heidi teaches an AVM calibration method by use of generative artificial intelligence by a computer device (paragraph 57, Heidi teaches a image system for calibration around the vehicle and cameras using markers.), comprising:
acquiring a camera image for an AVM video in a calibration facility space where markers are installed (Figure 1, paragraph 57-paragraph 60, Heidi teaches capturing images of the check shape markers.);
PNG
media_image1.png
702
529
media_image1.png
Greyscale
initially forming a marker candidate group by extracting a plurality of marker candidate images through outline-based image analysis of the camera image for each marker (paragraph 49, Heidi teaches capturing the chessboard markers and other suitable calibration patterns.); and
performing full-automatic-mode AVM calibration based on marker outlines for the marker candidate group by use of the generative AI model (paragraph 33-paragraph 40, paragraph 46-48, paragraph 98-paragraph 100, Heidi teaches using image processing and model information for calibrating the cameras using the marker data.)
Heidi does not explicitly disclose the following initializing a camera parameter (Z) for AVM video generation;
wherein the performing full-automatic-mode AVM calibration includes:
performing, by a policy network, adjustment of a camera parameter (Z) and primary filtering on the marker candidate group by a multilayer perceptron (MLP) neural network with individual marker candidate images as references;
performing, by a value network, secondary filtering on the marker candidate group by a convolutional neural network (CNN) with marker candidate images (hereinafter referred to as ‘common marker candidate image’) overlapping each other in a common region of adjacent cameras as references; and
performing, by a control network, additional adjustment of the camera parameter (Z) based on a marker probability value for the marker candidate group by an MDN-RNN neural network.
Chakravarty is in the same field of art of image processing for camera calibration. Further, Chakravarty teaches initializing a camera parameter (Z) for AVM video generation (paragraph 15, Chakravarty teaches capturing video data.);
wherein the performing full-automatic-mode AVM calibration includes:
performing, by a policy network, adjustment of a camera parameter (Z) and primary filtering on the marker candidate group by a multilayer perceptron (MLP) neural network with individual marker candidate images as references (paragraph 68-paragraph 70, Chakravarty teaches calibration of the image and fiducial marker within the image.);
performing, by a value network, secondary filtering on the marker candidate group by a convolutional neural network (CNN) with marker candidate images (hereinafter referred to as ‘common marker candidate image’) overlapping each other in a common region of adjacent cameras as references (paragraph 68-paragraph 70, Chakravarty teaches calibration of the image and fiducial marker within the image, the Examiner interprets that multiple filter points are being removed regarding the fiducial marker.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Heidi by incorporating the video capturing and the neural network for image analysis in the post processing of the data that is taught by Chakravarty, to make the invention that capture images around the camera of the car to determine the markers for camera calibration and using the neural network image processing for the data; thus, one of ordinary skilled in the art would be motivated to combine the references since Operation of a system can be supported by acquiring accurate and timely data regarding objects in a system's environment; and Generating the stitched image 700 can improve training of a first DNN 800 to determine the 4DoF pose of the vehicle 105 included in the stitched image 700. (paragraph 1, paragraph 76, Chakravarty).
Heidi in view of Chakravarty does not explicitly disclose performing, by a control network, additional adjustment of the camera parameter (Z) based on a marker probability value for the marker candidate group by an MDN-RNN neural network.
Ren is in the same field of art of image processing using neural networks. Further, Ren teaches performing, by a control network, additional adjustment of the camera parameter (Z) based on a marker probability value for the marker candidate group by an MDN-RNN neural network (paragraph 43, Ren teaches determining a confidence score for points and pixels.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Heidi in view of Chakravarty by the probability of the neural network data that is taught by Ren, to make the invention that capture images around the camera of the car to determine the markers for camera calibration and using the neural network image processing for the data that uses the neural networks for accurately determining the data; thus, one of ordinary skilled in the art would be motivated to combine the references since in another aspect, a calibrated sensor may be used in conjunction with a ground truth data collection tool to generate ground truth gaze data which may be used, for example, to train a machine learning model (e.g., a DNN) to perform operations, such as but not limited to, gaze detection (paragraph 4).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 8, Heidi teaches a non-transitory computer program contained in a non-transitory computer-readable storage medium comprising program code instructions which execute a AVM calibration method by use of generative artificial intelligence by a computer hardware device (paragraph 57, Heidi teaches a image system for calibration around the vehicle and cameras using markers.), the method comprising:
acquiring a camera image for an AVM video in a calibration facility space where markers are installed (Figure 1, paragraph 57-paragraph 60, Heidi teaches capturing images of the check shape markers.);
PNG
media_image1.png
702
529
media_image1.png
Greyscale
initially forming a marker candidate group by extracting a plurality of marker candidate images through outline-based image analysis of the camera image for each marker (paragraph 49, Heidi teaches capturing the chessboard markers and other suitable calibration patterns.); and
performing full-automatic-mode AVM calibration based on marker outlines for the marker candidate group by use of the generative AI model (paragraph 33-paragraph 40, paragraph 46-48, paragraph 98-paragraph 100, Heidi teaches using image processing and model information for calibrating the cameras using the marker data.).
Heidi does not explicitly disclose the following initializing a camera parameter (Z) for AVM video generation;
wherein the performing full-automatic-mode AVM calibration includes:
performing, by a policy network, adjustment of a camera parameter (Z) and primary filtering on the marker candidate group by a multilayer perceptron (MLP) neural network with individual marker candidate images as references;
performing, by a value network, secondary filtering on the marker candidate group by a convolutional neural network (CNN) with marker candidate images (hereinafter referred to as ‘common marker candidate image’) overlapping each other in a common region of adjacent cameras as references; and
performing, by a control network, additional adjustment of the camera parameter (Z) based on a marker probability value for the marker candidate group by an MDN-RNN neural network.
Chakravarty is in the same field of art of image processing for camera calibration. Further, Chakravarty teaches initializing a camera parameter (Z) for AVM video generation (paragraph 15, Chakravarty teaches capturing video data.);
wherein the performing full-automatic-mode AVM calibration includes:
performing, by a policy network, adjustment of a camera parameter (Z) and primary filtering on the marker candidate group by a multilayer perceptron (MLP) neural network with individual marker candidate images as references (paragraph 68-paragraph 70, Chakravarty teaches calibration of the image and fiducial marker within the image.);
performing, by a value network, secondary filtering on the marker candidate group by a convolutional neural network (CNN) with marker candidate images (hereinafter referred to as ‘common marker candidate image’) overlapping each other in a common region of adjacent cameras as references (paragraph 68-paragraph 70, Chakravarty teaches calibration of the image and fiducial marker within the image, the Examiner interprets that multiple filter points are being removed regarding the fiducial marker.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Heidi by incorporating the video capturing and the neural network for image analysis in the post processing of the data that is taught by Chakravarty, to make the invention that capture images around the camera of the car to determine the markers for camera calibration and using the neural network image processing for the data; thus, one of ordinary skilled in the art would be motivated to combine the references since Operation of a system can be supported by acquiring accurate and timely data regarding objects in a system's environment; and Generating the stitched image 700 can improve training of a first DNN 800 to determine the 4DoF pose of the vehicle 105 included in the stitched image 700. (paragraph 1, paragraph 76, Chakravarty).
Heidi in view of Chakravarty does not explicitly disclose performing, by a control network, additional adjustment of the camera parameter (Z) based on a marker probability value for the marker candidate group by an MDN-RNN neural network.
Ren is in the same field of art of image processing using neural networks. Further, Ren teaches performing, by a control network, additional adjustment of the camera parameter (Z) based on a marker probability value for the marker candidate group by an MDN-RNN neural network (paragraph 43, Ren teaches determining a confidence score for points and pixels.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Heidi in view of Chakravarty by the probability of the neural network data that is taught by Ren, to make the invention that capture images around the camera of the car to determine the markers for camera calibration and using the neural network image processing for the data that uses the neural networks for accurately determining the data; thus, one of ordinary skilled in the art would be motivated to combine the references since in another aspect, a calibrated sensor may be used in conjunction with a ground truth data collection tool to generate ground truth gaze data which may be used, for example, to train a machine learning model (e.g., a DNN) to perform operations, such as but not limited to, gaze detection (paragraph 4).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Pertinent Art
The prior art made of record and not relied upon is considered pertinent to the applicant’s disclosure.
Kothari et al U.S. Patent Publication No. 20230401745.
Maruno et al U.S. Patent Publication No. 2023/0388477.
Liu et al U.S. Patent Publication No. 2023/0283906.
Okouneva et al U.S. Patent Publication No. 2023/0202389.
Nagai et al U.S. Patent Publication No. 2020/0175722.
Allowable Subject Matter
Claim 9 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action.
Claims 2-7 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ONEAL R MISTRY whose telephone number is (313)446-4912. The examiner can normally be reached on 9am-5pm.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ONEAL R MISTRY/
Examiner, Art Unit 2665
Prosecution Insights