DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
A Preliminary Amendment was made 03/18/2024 to amend the specification and drawings.
Election/Restrictions
Applicant’s election without traverse of Claims 1-12 in the reply filed on April 28, 2026 is acknowledged.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on March 18, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is considered by examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: Object detection based on image data recognizing object motion.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such limitations are: Claim 1: “generation unit” and “detection unit”. Claim 11: “generation step” and “detection step”. Claim 12: “generation unit” and “detection unit”.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, each are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
The following structure, function and linking of structure to function was identified:
Regarding Claims 1 and 12 (claimed in parallel):
“generation unit” is identified as element 301 of the Detection System 300, shown in Fig 3, 20 and the specific algorithm steps for generating the sensing image on a basis of sensor data including speed of an object described in prose (specification ¶ [0107]-[0111], [0125]-[0127], [0131]-[0134] and shown in Fig 4-6, 11, 14-16).
“detection unit” is identified as element 302 of the Detection System 300 shown in Fig 3, 20, structurally described as a CNN and described in prose, including the DNN (specification ¶ [0114]-[0121], [0128]-[0129], [0136]-[0137] and Figures 7-10, 12-13, 17-19) and the training of the DNN (specification ¶ [0145]-[0154] and Fig 21-23).
Regarding Claim 11:
“generation step” is identified as S2002 Fig 20, S2403 Fig 24 and specification ¶ [0139]-[0141], with further reference of processing referred back to the “generation unit” (see specification citations above).
“detection step” is identified as S2004 Fig 20, S2404 Fig 24 and specification ¶ [0142]-[0143], with further reference of processing referred back to the “detection unit” (see specification citations above).
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because data per se and/or computer programs do not fall into one of the four categories of statutory invention (machine, process, manufacture, composition). More specifically, claims are eligible for patent protection under § 101 if they are in one of the four statutory categories and not directed to a judicial exception to patentability (i.e., laws of nature, natural phenomena, and abstract ideas). Alice Corp. v. CLS Bank Int'l, 573 U. S. 208 (2014)
Regarding claim 12, the claim is drawn towards a “computer program”. As described in MPEP § 2106, data per se and computer programs do not fall into one of the four statutory categories. Therefore, since claim 12 is drawn towards a program, the claim is not eligible for patent protection.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 9-12 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Liu et al (JP 2020-16597, with foreign priority cited as application JP 2018-140973 in US application US 2021/0311169 with the US application cited in the rejections herein).
Regarding Claim 1, Liu et al teach an information processing apparatus (object determination device 1; Fig 1 and ¶ [0049]) comprising:
a generation unit (image generator 32; Fig 1 and ¶ [0053]) that generates a sensing image on a basis of sensor data including speed information of an object (data generated by the radar apparatus 2 is millimeter waveband data and input to the object determination device 1 including to the image generator 32 of data processing controller 22 to generate an image; Fig 1 and ¶ [0051], [0055]); and
a detection unit (object determiner 23; Fig 1 and ¶ [0056]) that detects the object from the sensing image using a learned model (object determiner 23 using a deep learning model to detect the object from the image data; Fig 1 and ¶ [0056]).
Regarding Claim 2, Liu et al teach the information processing apparatus according to claim 1 (as described above), wherein the detection unit (object determiner 23; Fig 1 and ¶ [0056]) performs object detection using the learned model learned to recognize the object included in the sensing image (object determiner 23 using a deep learning model to detect the object from the image data; Fig 1 and ¶ [0056]).
Regarding Claim 9, Liu et al teach the information processing apparatus according to claim 1 (as described above), wherein the learned model includes a DNN (the object determiner 23 using a deep learning model, in particular a convolutional neural network; Fig 1 and ¶ [0064]).
Regarding Claim 10, Liu et al teach the information processing apparatus according to claim 1 (as described above), wherein the sensor data is data captured by at least one sensor of a millimeter wave radar, a LiDAR, or a sound wave sensor (data generated by the radar apparatus 2 is millimeter waveband data and input to the object determination device 1 including to the image generator 32 of data processing controller 22 to generate an image; Fig 1 and ¶ [0051], [0055]).
Regarding Claim 11, Liu et al teach an information processing method (process to perform operations of object determination device 1; Fig 1 and ¶ [0049], [0054]-[0055]) comprising:
a generation step (process to use image generator 32; Fig 1 and ¶ [0053]) of generating a sensing image on a basis of sensor data including speed information of an object (data generated by the radar apparatus 2 is millimeter waveband data and input to the object determination device 1 including to the image generator 32 of data processing controller 22 to generate an image; Fig 1 and ¶ [0051], [0055]); and
a detection step (process to use object determiner 23; Fig 1 and ¶ [0056]) of detecting the object from the sensing image using a learned model (object determiner 23 using a deep learning model to detect the object from the image data; Fig 1 and ¶ [0056]).
Regarding Claim 12, Liu et al teach a computer program written in a computer-readable format to cause a computer to (program stored in storage 13 and implemented by a processor to perform operations of object determination device 1; Fig 1 and ¶ [0049], [0054]-[0055]) function as:
a generation unit (image generator 32; Fig 1 and ¶ [0053]) that generates a sensing image on a basis of sensor data including speed information of an object (data generated by the radar apparatus 2 is millimeter waveband data and input to the object determination device 1 including to the image generator 32 of data processing controller 22 to generate an image; Fig 1 and ¶ [0051], [0055]); and
a detection unit (object determiner 23; Fig 1 and ¶ [0056]) that detects the object from the sensing image using a learned model (object determiner 23 using a deep learning model to detect the object from the image data; Fig 1 and ¶ [0056]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3-8 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (JP 2020-16597, with foreign priority cited as application JP 2018-140973 in US application US 2021/0311169 with the US application cited in the rejections herein) in view of Wekel et al (US 2021/0063578, cited in IDS 03/18/2024).
Regarding Claim 3, Liu et al teach the information processing apparatus according to claim 1 (as described above), including the generation unit (image generator 32; Fig 1 and ¶ [0053]).
Liu et al does not teach to projects sensor data including a three-dimensional point cloud onto a two- dimensional plane to generate the sensing image.
Wekel et al is analogous art pertinent to the technological problem addressed in the current application and teaches to projects sensor data including a three-dimensional point cloud onto a two- dimensional plane to generate the sensing image (DNN 126 uses a 3D LiDAR point cloud data projected onto a 2D plane of the range image 102; Fig 1 and ¶ [0032], [0034]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Liu et al with Wekel et al including to projects sensor data including a three-dimensional point cloud onto a two- dimensional plane to generate the sensing image. By projecting LiDAR point cloud data from a 3D plane to a 2D plane, multiple visual data sources may be combined to enhance data used to perceive an environment thereby enabling an autonomous vehicle to more safely operate, as recognized by Wekel et al (¶ [0003]-[0004]).
Regarding Claim 4, Liu et al in view of Wekel et al teach the information processing apparatus according to claim 3 (as described above), wherein the generation unit generates the sensing image having a pixel value corresponding to the speed information (Wekel et al, the LiDAR sensors 864 generate data points reflective of speed of objects in relation to the LiDAR sensor, which is reflected in the range image 102; ¶ [0070]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Liu et al with Wekel et al including wherein the generation unit generates the sensing image having a pixel value corresponding to the speed information. By using LiDAR data, speed data of objects may be detected and calculated, thereby providing information about an environment surrounding a vehicle and allowing for an intelligent system to more safely operate, as recognized by Wekel et al (¶ [0003]-[0004]).
Regarding Claim 5, Liu et al in view of Wekel et al teach the information processing apparatus according to claim 4 (as described above), wherein the generation unit divides one sensing image into a plurality of sub-images on a basis of the pixel value (Wekel et al, a segmentation mask is used for identifying the LiDAR point cloud data corresponding to the given unique actor instance based on each unique object instance determined based on the LiDAR pixel value (each object is identified with an individual mask (sub-image)); Fig 1-2B and ¶ [0036]-[0037]), and the detection unit inputs the plurality of sub-images to the learned model to detect the object (Wekel et al, each unique object instance having its own segmentation mask (sub-image) is classified based on its characterized features in the DNN 126; Fig 1-2B and ¶ [0036]-[0037]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Liu et al with Wekel et al including wherein the generation unit divides one sensing image into a plurality of sub-images on a basis of the pixel value, and the detection unit inputs the plurality of sub-images to the learned model to detect the object. By using specific masks for each object, individual data may be generated for each object and allow for specific classification and features associated with the object with accuracy, as recognized by Wekel et al ( [0036]-[0037]).
Regarding Claim 6, Liu et al in view of Wekel et al teach the information processing apparatus according to claim 5 (as described above), wherein the detection unit inputs a time series of each of sub images divided from each of a plurality of consecutive sensing images to the learned model to detect the object (Wekel et al, the LiDAR range images are accumulated over time (¶ [0074]) and analyzed by the DNN to detect, classify and track over time the given detected object across frames; Fig 1-2B and ¶ [0036]- [0037]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Liu et al with Wekel et al including wherein the detection unit inputs a time series of each of sub images divided from each of a plurality of consecutive sensing images to the learned model to detect the object. By using data generated over time, unique actor instances may be identified and tracked across frames, thereby allowing for movements and patterns to be generated and used to aid in path planning, obstacle or collision avoidance and other operations of a vehicle, as recognized by Wekel et al (¶ [0036]).
Regarding Claim 7, Liu et al in view of Wekel et al teach the information processing apparatus according to claim 5 (as described above), wherein the detection unit performs object detection using the learned model learned to recognize the object from the plurality of sub-images obtained by dividing the sensing image on a basis of the pixel value (Wekel et al, the DNN 126 is trained to detect individual objects and classes based on the pixel value, representing the detected LiDAR point data value; Fig 1-2D and ¶ [0036]-[0037], [0045]-[0046]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Liu et al with Wekel et al including wherein the detection unit performs object detection using the learned model learned to recognize the object from the plurality of sub-images obtained by dividing the sensing image on a basis of the pixel value. By training the DNN using LiDAR data of similar surroundings, the DNN is trained similarly to how it performs during inference, thereby allowing for real-world images to match similarly to training data, without requiring more training than necessary to optimize the DNN, as recognized by Wekel et al (¶ [0045]-[0046]).
Regarding Claim 8, Liu et al in view of Wekel et al teach the information processing apparatus according to claim 5 (as described above), wherein the generation unit adds a texture corresponding to the speed information to each of the sub-images (Wekel et al, the DNN 126 analyzes the range image 102 to compute/infer texture of the incident object as reflective of the LiDAR data points; Fig 2B and ¶ [0034]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Liu et al with Wekel et al including wherein the generation unit adds a texture corresponding to the speed information to each of the sub-images. By using a texture for a given object, a visual represented is encoded to associate a pixel with a given intensity of a LiDAR point, thereby allowing for visual cue in data representation, as recognized by Wekel et al (¶ [0034]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Achour et al (US 2018/0348343) teach a system and method for identifying an object using radar that considers positional data and speed from the vehicle sensor.
Rohani et al (US 10,353,053) teach a method and system for using radar systems for object detection based on machine learning including detecting dynamic and static objects by a vehicle.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATHLEEN M BROUGHTON whose telephone number is (571)270-7380. The examiner can normally be reached Monday-Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATHLEEN M BROUGHTON/Primary Examiner, Art Unit 2661
Prosecution Insights