DETAILED ACTION
This is a first action on the merits. Claims 1-14 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement submitted on 12/21/2024 has been reviewed and considered.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference characters "320" (para. 95, l. 1) and "328" (para. 95, l. 2) have both been used to designate the maneuvering system. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “320” has been used to designate both sensor system (para. 92, l. 1) and maneuvering system (para. 95, l. 1). Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The abstract of the disclosure is objected to because, in line 1, it contains the implied phrase “The present invention relates to”. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
The disclosure is objected to because of the following informalities:
In paragraph 10, lines 1-2, “the disclosed technology may be that it provides a computationally favorable method” should read “the disclosed technology may provide a computationally favorable method”. This appears to be a typographical error.
In paragraph 28, lines 2-3, “embodied in apparatus comprising one or more processors, one or more memories” should read “embodied in an apparatus comprising one or more processors, and one or more memories”. This appears to be a typographical error.
In paragraph 29, line 1, “for purpose of describing” should read “for the purpose of describing”. This appears to be a typographical error.
In paragraph 34, lines 1-2, “parts of a road … thus is free” should read “parts of road … thus are free”. This appears to be a typographical error.
In paragraph 36, lines 1-2, “parts of a road that is free” should read “parts of a road that are free”. This appears to be a typographical error.
In paragraph 54, line 3, “provide further signal” should read “provide further signals”. This appears to be a typographical error.
In paragraph 66, lines 1-2, “By determining the motion relative the ground allows for differentiating dynamic objects” should read “Determining the motion relative the ground allows for differentiating dynamic objects”. This appears to be a typographical error.
In paragraph 66, lines 2-3, “objects which has a motion” should read “objects which have a motion”. This appears to be a typographical error.
In paragraph 66, lines 5-6, “objects which has motion relative the ego vehicle … but is in stand-still” should read “objects which have motion relative the ego vehicle … but are at a standstill”. This appears to be a typographical error.
In paragraph 68, line 2, it is unclear if the reference to figure 4 is directed towards one or both of figures 4A and 4B.
In paragraph 68, line 8, “the boarder between” should read “the border between”. This appears to be a typographical error.
In paragraph 90, line 14, the acronyms DRAM, SRAM and DDR RAM should be defined.
In paragraph 93, line 14, the acronym ETSI should be defined.
In paragraph 94, line 5, the acronyms CAN and I2C should be defined.
In paragraph 94, line 7, the terms WIFI, LORA, ZIGBEE, and BLUETOOTH, which are a trade name or a mark used in commerce, should be capitalized wherever they appear or, where appropriate, include a proper symbol indicating use in commerce such as ™, SM, or ® following the term. Although the use of trade names and marks used in commerce (i.e., trademarks, service marks, certification marks, and collective marks) are permissible in patent applications, the proprietary nature of the marks should be respected and every effort made to prevent their use in any manner which might adversely affect their validity as commercial marks.
In paragraph 110, line 1, “the cells … is herein depicted” should read “the cells … are herein depicted”. This appears to be a typographical error.
Appropriate correction is required.
Claim Objections
Claims 1 and 12 are objected to because of the following informalities:
Claims 1 and 12 should each be limited to a single colon because using multiple colons in a single sentence is grammatically incorrect which makes it difficult to determine the hierarchical relationship between each limitation.
In claim 12, line 2, “an image capturing device of the vehicle” should read “an image capturing device of a vehicle” to provide sufficient antecedent basis for the vehicle in the claim.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a trajectory planning module configured to generate candidate trajectories” in claims 10 and 13, lines 2 and 2-3, respectively.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 1 and 12, lines 12 and 11, respectively, the limitation “the image plane of an image” renders the claim indefinite because it lacks sufficient antecedent basis in the claim, therefore, it is unclear if the image has one plane, or a plurality of planes that pass through the 3D points at different depths. The image plane of the first image is rendered further unclear by reciting “an image plane of the subsequent image” in lines 14-15. Paragraph 57 appears to disclose a 2D image has a single image plane that is the 2D coordinate system spanned by it two axes, therefore, for the purposes of examination, it will be assumed that each image has a single image plane.
Regarding claim 3, line 2, the limitation “determine a depth map” renders the claim indefinite because it is unclear if the depth map is the depth map used for obtaining a set of 3D points in claim 1, line 6. Paragraph 52 appears to disclose the depth map based on an output of a machine learning model is used to obtain the set of 3D points, therefore, for the purposes of examination, it will be assumed that claim 3 is directed to determining the depth map of claim 1.
Regarding claim 3, lines 2-3, the limitation “based on an image sequence” renders the claim indefinite because it is unclear if the depth map is based on the image sequence captured in claim 1, line 3. Paragraph 52 discloses a machine learning model may determine a depth map using an image sequence, and further discloses the machine learning model may be trained using images. Paragraph 53 further discloses the depth map may be based on a LIDAR point cloud of the scene, and obtaining the image sequence and the set of 3D points may be performed concurrently as the vehicle travels along a road. Paragraphs 52-53 appear to disclose images used in advance to train the machine learning to determine the depth map, as well as generating the depth map using real-time images from the vehicle, therefore, for the purposes of examination, it will be assumed that claim 3 is directed to determining a depth map using either real-time or pre-recorded images.
Regarding claim 5, lines 1-2, the limitation “the motion data is indicative of an estimated motion of the object” renders the claim indefinite because it is unclear if the estimated motion of the object is the same motion estimated in claim 1, line 10. Paragraphs 63-64 appear to disclose different embodiments include estimating the motion of the object relative to the vehicle or the ground, therefore, for the purposes of examination, it will be assumed that claims 1 and 5 are directed to estimating the same motion.
Regarding claim 6, line 3, the limitation “the motion data is further indicative of an estimated motion of the object” renders the claim indefinite because it is unclear if the estimated motion of the object is the same motion estimated in claim 1, line 10. Paragraphs 63-64 appear to disclose different embodiments include estimating one of the motion of the object relative to the vehicle or the ground, therefore, for the purposes of examination, it will be assumed that claims 1 and 5 are directed to estimating the same motion.
Regarding claims 10 and 13, lines 2 and 2-3, respectively, the limitation “a trajectory planning module configured to generate candidate trajectories” invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. Paragraph 100 discloses the ADS may comprise a path planning module, also referred to as a trajectory planning module, however, no specific structure is defined for implementing the ADS or its modules. Paragraph 102 further discloses parts of the solution may be implemented in the vehicle, in a system external to the vehicle, or a combination thereof. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph. For the purposes of examination, it will be assumed that the trajectory planning module comprises instructions executable by a processor, and equivalents thereof.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Regarding claim 14, lines 1-3, the limitation “A vehicle … comprising: an image capturing device, and a device” renders the claim indefinite because it is unclear if the vehicle, image capturing device, and device are the same elements recited in claim 12, lines 1-2. For the purposes of examination, it will be assumed that claims 12 and 14 are directed to the same vehicle, image capturing device, and device.
Claims 2-11 and 13-14 are rejected as being dependent on a rejected claim and for failing to cure the deficiencies listed above.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
The determination of whether a claim recites patent ineligible subject matter is a two-step inquiry.
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), see MPEP § 2106.03, or
STEP 2: the claim recites a judicial exception, e.g., an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis: see MPEP § 2106.04
STEP 2A (PRONG ONE): Does the claim recite an abstract idea, law of nature, or natural phenomenon? see MPEP § 2106.04(II)(A)(1)
STEP 2A (PRONG TWO): Does the claim recite additional elements that integrate the judicial exception into a practical application? see MPEP § 2106.04(II)(A)(2)
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? see MPEP § 2106.05
Claims 1-14 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
101 Analysis – Step 1
Claim 12 is directed to a device (i.e., a machine). Therefore, claim 12 is within at least one of the four statutory categories.
101 Analysis – Step 2A, Prong One
Regarding Prong One of the Step 2A analysis, the claims are to be analyzed to determine whether they recite subject matter that falls within one of the following groups of abstract ideas: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental processes. see MPEP § 2106(A)(II)(1) and MPEP § 2106.04(a)-(c). Independent claim 12 includes limitations that recite an abstract idea (emphasized below [with the category of abstract idea in brackets]) and will be used as a representative claim for the remainder of the analysis. Claim 12 recites:
A device comprising control circuitry configured to:
obtain an image sequence captured by an image capturing device of the vehicle, wherein the image sequence comprises a plurality of images depicting a scene at a respective time instance of a plurality of time instances;
obtain a set of 3D points based on a depth map of the scene depicted in the image sequence, wherein each 3D point of the set of 3D points is associated with a three-dimensional position of the 3D point within the scene;
determine motion data associated with each 3D point of the set of 3D points, wherein the motion data is indicative of an estimated motion of an object in the scene associated with the 3D point, wherein the motion data associated with each 3D point is determined by [mental process/step]:
obtaining a 2D point in the image plane of an image of the image sequence corresponding to the 3D point; applying an optical flow between the image and a subsequent image of the sequence of images, to the 2D point, thereby obtaining a subsequent 2D point in an image plane of the subsequent image [mental process/step];
determining a subsequent 3D point by projecting the subsequent 2D point based on the depth map of the scene; and determining the motion data based on a difference between the three-dimensional position of the 3D point and the subsequent 3D point [mental process/step]; and
assign the set of 3D points with associated motion data to a free-space estimation of the scene, based on the three-dimensional position associated with each 3D point [mental process/step].
The examiner submits that the foregoing bolded limitation(s) constitute a “mental process” because under its broadest reasonable interpretation, the claim covers performance of the limitation in the human mind. For example, “determine motion data…”, “…applying an optical flow…”, “determining a subsequent 3D point…”, and “assign the set of 3D points … to a free space estimation..” in the context of this claim encompasses a person tracking an objects position through a video. Accordingly, the claim recites at least one abstract idea.
101 Analysis – Step 2A, Prong Two
Regarding Prong Two of the Step 2A analysis, the claims are to be analyzed to determine whether the claim, as a whole, integrates the abstract idea into a practical application. see MPEP § 2106.04(II)(A)(2) and MPEP § 2106.04(d)(2). It must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.”
In the present case, the additional limitations beyond the above-noted abstract idea are as follows (where the underlined portions are the “additional limitations” [with a description of the additional limitations in brackets], while the bolded portions continue to represent the “abstract idea”):
A device comprising control circuitry configured to [applying the abstract idea using generic computer components]:
obtain an image sequence captured by an image capturing device of the vehicle, wherein the image sequence comprises a plurality of images depicting a scene at a respective time instance of a plurality of time instances [pre-solution activity (data gathering) using a generic sensor];
obtain a set of 3D points based on a depth map of the scene depicted in the image sequence, wherein each 3D point of the set of 3D points is associated with a three-dimensional position of the 3D point within the scene [pre-solution activity (receiving data)];
determine motion data associated with each 3D point of the set of 3D points, wherein the motion data is indicative of an estimated motion of an object in the scene associated with the 3D point, wherein the motion data associated with each 3D point is determined by:
obtaining a 2D point in the image plane of an image of the image sequence corresponding to the 3D point; applying an optical flow between the image and a subsequent image of the sequence of images, to the 2D point, thereby obtaining a subsequent 2D point in an image plane of the subsequent image;
determining a subsequent 3D point by projecting the subsequent 2D point based on the depth map of the scene; and determining the motion data based on a difference between the three-dimensional position of the 3D point and the subsequent 3D point; and
assign the set of 3D points with associated motion data to a free-space estimation of the scene, based on the three-dimensional position associated with each 3D point.
For the following reason(s), the examiner submits that the above identified additional limitations do not integrate the above-noted abstract idea into a practical application.
Regarding the additional limitation(s) of “obtain an image sequence…” and “obtain a set of 3D points…”, the examiner submits that the limitation(s) is/are insignificant extra-solution activities that merely use a computer (a device comprising control circuitry) to perform the process. In particular, the image sequence step is recited at a high level of generality (i.e., as a general means of receiving video), and amounts to merely gathering data using a generic sensor, which is a form of insignificant extra-solution activity. The 3D points step is recited at a high level of generality (i.e., as a general mean of receiving location data), and amounts to merely receiving data, which is a form of insignificant extra-solution activity. The “device comprising control circuitry” is/are also recited at a high level of generality (i.e., as generic computer components performing the generic computer function(s) of receiving and processing image data) such that it amounts to no more than mere instructions to apply the exception using a generic computer component.
Thus, taken alone, the additional elements do not integrate the abstract idea into a practical application. Further, looking at the additional limitation(s) as an ordered combination or as a whole, the limitation(s) add nothing that is not already present when looking at the elements taken individually. For instance, there is no indication that the additional elements, when considered as a whole, reflect an improvement in the functioning of a computer or an improvement to another technology or technical field, apply or use the above-noted judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, implement/use the above-noted judicial exception with a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing, or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is not more than a drafting effort designed to monopolize the exception. see MPEP § 2106.05. Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
101 Analysis – Step 2B
Regarding Step 2B of the Revised Guidance, representative independent claim 12 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a device comprising control circuitry to perform the “determin[ing] motion data…”, “…applying an optical flow…”, “determining a subsequent 3D point…”, and “assign the set of 3D points … to a free space estimation..” amounts to nothing more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Also discussed above with respect to integration of the abstract idea into a practical application, the examiner submits that the additional limitation(s) of “obtain an image sequence…” and “obtain a set of 3D points…” is/are insignificant extra-solution activities. Hence, the claim is not patent eligible.
Claim(s) 1 is/are substantially the same subject matter as claim 12 except drawn to a method (i.e., a process) which falls under one of the statutory categories in step 1. Therefore, claim(s) 1 is/are rejected under step 2 for the same reasons above.
Dependent claim(s) 2-11 and 13-14 do not recite any further limitations that cause the claim(s) to be patent eligible. Rather, the limitations of the dependent claims are directed toward additional aspects of the judicial exception. Although claims 10 and 13 disclose a trajectory planning module configured to generate candidate trajectories of the vehicle, the claims do not recite controlling the vehicle using one of the candidate trajectories. Therefore, dependent claims 2-11 and 13-14 are not patent eligible under the same rationale as provided for in the rejection of claims 1 and 12.
Therefore, claims 1-14 is/are ineligible under 35 U.S.C 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 7 and 9-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 11,744,983) in view of Kocamaz et al. (US 2023/0360255), hereinafter Zhang and Kocamaz, respectively.
Regarding claims 1, 12 and 14, as best understood, Zhang discloses a computer-implemented method, performed in a vehicle equipped with an automated driving system (Zhang; col. 5, ll. 32-34: the technology disclosed can be applied to autonomous vehicle guidance technology), the method comprising: obtaining an image sequence captured by an image capturing device of the vehicle, wherein the image sequence comprises a plurality of images depicting a scene at a respective time instance of a plurality of time instances (Zhang; col. 16, ll. 34-36: One feature extractor 1102 implementation employs optical flow methods to calculate the motion between two image frames, taken at times t and t+Δt at each voxel position.); obtaining a set of 3D points based on a depth map of the scene depicted in the image sequence, wherein each 3D point of the set of 3D points is associated with a three-dimensional position of the 3D point within the scene (Zhang; col. 49, ll. 2-4: where a depth sensor is used as an auxiliary sensor, a depth map is also received at the Control Unit); determining motion data associated with each 3D point of the set of 3D points, wherein the motion data is indicative of an estimated motion of an object in the scene associated with the 3D point, wherein the motion data associated with each 3D point is determined by: obtaining a 2D point in the image plane of an image of the image sequence corresponding to the 3D point (Zhang; col. 16, ll. 13-14: Optical flow gives 2D-2D correspondence between previous image and a current image.); applying an optical flow between the image and a subsequent image of the sequence of images, to the 2D point, thereby obtaining a subsequent 2D point in an image plane of the subsequent image (Zhang; col. 25, ll. 46-47: At step 1510, a new frame is captured by the camera at the new location.; col. 24, ll. 59-64: Based on the current image observation, the information needed for propagation, e.g., features, poses, map points, etc. is prepared. Then 2D-3D correspondence information for the optical flow tracked 2D features is obtained by directly using 2D-2D correspondences from optical flow tracking results.); determining a subsequent 3D point by projecting the subsequent 2D point based on the depth map of the scene (Zhang; col. 25, ll. 58-65: At step 1540, depth values for the remaining features on the list of features are retrieved from the table of depth values. At step 1545, a weighted average depth value is calculated for each remaining feature in the feature list with the depth values of all the available pixel coordinates for that feature. At step 1550, a 3D map is populated with the remaining list of features on the list of the features with their corresponding depth value.); and assigning the set of 3D points with associated motion data to a free-space estimation of the scene, based on the three-dimensional position associated with each 3D point (Zhang; col. 33; ll. 47-54: Monocular-auxiliary sensor prepares an occupancy map 1955 by reprojecting feature points 1901, 1911, 1941, 1951, 1922 onto a 2D layer corresponding to the floor of the room 1900. In some implementations, second and possibly greater occupancy maps are created at differing heights of the robot 1925, enabling the robot 1925 to navigate about the room 1900 without bumping its head into door soffits, or other obstacles above the floor.).
Although Zhang discloses determining motion data associated with the 3D points by determining the optical of the 2D points, Zhang does not appear to explicitly disclose determining the motion data based on a difference between the three-dimensional position of the 3D point and the subsequent 3D point.
Kocamaz, in the same field of endeavor (autonomous vehicle image processing), discloses determining motion data based on a difference between a three-dimensional position of a 3D point and a subsequent 3D point (Kocamaz; para. 107: The method 1100, at block B1106, may include determining, based at least on the first 3D detected location, a 3D predicted location associated with the tracked object. For instance, the tracking component 108 may determine the 3D predicted location associated with the tracked object at the second instance in time. In some examples, to determine the 3D predicted location, the tracking component 108 may use the first 3D detected location along with one or more states associated with the tracked object, such as the velocity, the acceleration, the orientation, and/or the like.).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have determined, using the 3D locations, the velocity and acceleration of the surrounding detected objects, as disclosed by Kocamaz, in the control unit of Zhang, to yield the predictable result of accurately determining the trajectory of an object that could impact the vehicle.
Regarding claim 7, as best understood, Zhang, as modified, discloses the motion data comprises information pertaining to a velocity of the object in the scene (Kocamaz; para. 67: the prediction component 110 may further use the state data 112 to determine additional state information, such as a predicted shape, velocity, acceleration, and/or the like of the object).
Regarding claim 9, as best understood, Zhang, as modified, discloses assigning the set of 3D points with associated motion data to the free-space estimation of the scene comprises: selecting a subset of the set of 3D points corresponding to a free-space boundary of the free-space estimation (Zhang; col. 33, ll. 25-28: Monocular-auxiliary sensor determines feature points 1901, 1911, 1941, 1951, 1922, and so forth for the walls, corners and door 1923 of room 1900 from the information in the captured image frames.), and assigning aggregated motion data to the free-space boundary based on the subset of 3D points (Kocamaz; para. 63: prediction component 110 may use the bounding shape 302 and a transition vector, where the transition vector includes at least the final scalar change and the final translation, to determine a new bounding shape 416 (e.g., a predicted and/or estimated location) for the object).
Regarding claims 10 and 13, as best understood, Zhang, as modified, discloses providing the free-space estimation to a trajectory planning module configured to generate candidate trajectories of the vehicle (Zhang; col. 34, ll. 13-17: the robot uses the occupancy grid 1955 in order to plan a trajectory 1956 from its current location to another location in the map using the technology described herein above in the Mapping sections).
Regarding claim 11, as best understood, Zhang, as modified, discloses a non-transitory computer readable storage medium storing instructions, which when executed by a computing device, causes the computing device to carry out the method according to claim 1 (Zhang; para. 49, ll. 22-24: a non-transitory computer readable storage medium storing instructions executable by a processor to perform any of the methods described above).
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Kocamaz as applied to claim 1 above, and further in view of Pazhayampallil et al. (US 2021/0261157), hereinafter Pazhayampallil.
Regarding claim 2, as best understood, Zhang, as modified, discloses determining the motion data further comprises obtaining an estimation of a ground plane of the scene depicted in the sequence of images (Zhang; para. 29, ll. 32-39: The occupancy grid map can be a plane oriented substantially perpendicular to the direction of gravity. Each layer has a specific height. (The layer on the floor is typically 0 according to one convention). FIG. 17 illustrates an example of an occupancy grid map in one implementation. Occupancy grid 1700 of FIG. 17 indicates a single layer, such as a floor layer 1702, mapped by an implementation of the monocular-auxiliary sensor.; para. 32, ll. 10-12: Obtain at least 3 points on the ground-plane from the descriptive point cloud to estimate the ground-plane's normal in the original coordinate.).
It is unclear if Zhang, as modified, explicitly discloses the motion data is determined as a motion parallel to the estimated ground plane.
However, Pazhayampallil, in the same field of endeavor (autonomous vehicle collision avoidance), explicitly discloses motion data is determined as a motion parallel to an estimated ground plane (Pazhayampallil; para. 82: Generally, motion of ground-based objects (e.g., vehicles, pedestrians), may occur approximately with in a horizontal plane (i.e., parallel to a ground plane), including linear motion along an x-axis, linear motion along a y-axis, and rotation about a z-axis normal to the horizontal plane, which may be represented as a linear velocity in the horizontal plane and an angular velocity about an axis normal to the horizontal plane. This variation of the method S100 is thus described below as executed by the autonomous vehicle to derive tangential, angular, and total velocities of an object within a horizontal plane given radial velocities and positions (e.g., ranges and angles) of points on the object in the horizontal plane.).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have determined, with respect to the occupancy grid map layers, the velocity and acceleration of the surrounding detected objects in the control unit of Zhang as modified, by projecting their motion onto a horizontal plane parallel to a ground plane, as disclosed by Pazhayampallil, to yield the predictable result of ignoring object trajectories that will not intersect with the vehicle's path.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Kocamaz as applied to claim 1 above, and further in view of McIntosh (US 2025/0054167).
Regarding claim 3, as best understood, Zhang, as modified, discloses the invention substantially as claimed as describe above.
Zhang, as modified, does not explicitly disclose the depth map is based on an output of a machine learning model configured to determine a depth map based on an image sequence as input..
McIntosh, in the same field of endeavor (autonomous vehicle image processing), discloses a depth map is based on an output of a machine learning model configured to determine a depth map based on an image sequence as input (McIntosh; para. 23: a machine learning (ML) model is used to generate a dense depth map based on one or more frames/images of a video).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have modified the depth map generated by the depth sensor of Zhang, as modified, to be generated from the images by a machine learning model, as disclosed by McIntosh, to yield the predictable result of accurately determining the depth information when there are many objects in the surrounding environment.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Kocamaz as applied to claim 1 above, and further in view of Shugrina et al. (US 12,555,260), hereinafter Shugrina.
Regarding claim 4, as best understood, Zhang, as modified, discloses the invention substantially as claimed as described above.
Although Zhang, as modified, discloses the imaging system can use LiDAR (Zhang; col. 5, ll. 63-67), Zhang, as modified, does not appear to explicitly disclose the depth map is based on a LIDAR point cloud of the scene.
Shugrina, in the same field of endeavor (autonomous vehicle navigation), discloses a depth map is based on a LIDAR point cloud of the scene (Shugrina; col. 5, ll. 19-20: a depth map for an image is measured with one or more LiDAR sensors when said image is captured).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have modified the depth map generated by the depth sensor of Zhang, as modified, to be generated from LiDAR measurements, as disclosed by Shugrina, to yield the predictable result of accurately measuring distances to the objects in the surrounding environment.
Claim(s) 5-6 and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Kocamaz as applied to claim 1 above, and further in view of Nastroshvili et al. (US 2019/0049580), hereinafter Nastroshvili.
Regarding claim 5, as best understood, Zhang, as modified, discloses the motion data is indicative of an estimated motion of the object in the scene associated with the 3D point (Kocamaz; para. 67: the prediction component 110 may further use the state data 112 to determine additional state information, such as a predicted shape, velocity, acceleration, and/or the like of the object).
It is unclear if Zhang, as modified, explicitly discloses estimating motion of the object relative a motion of the vehicle.
However, Natroshvili, in the same field of endeavor (autonomous vehicle navigation), discloses estimating motion of an object relative a motion of a vehicle (Natroshvili; para. 56: the velocity information associated with sensor measurements may be relative to the velocity of the ego vehicle).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have determined the velocity of the surrounding detected objects in the control unit of Zhang, as modified, relative to the vehicle's velocity, as disclosed by Natroshvili, to yield the predictable result of accurately determining the approach speed of an object that may collide with the vehicle.
Regarding claim 6, as best understood, Zhang, as modified, discloses the motion data is further indicative of an estimated motion of the object in the scene associated with the 3D reference point relative the ground (Zhang; para. 29, ll. 32-39: The occupancy grid map can be a plane oriented substantially perpendicular to the direction of gravity. Each layer has a specific height. (The layer on the floor is typically 0 according to one convention). FIG. 17 illustrates an example of an occupancy grid map in one implementation. Occupancy grid 1700 of FIG. 17 indicates a single layer, such as a floor layer 1702, mapped by an implementation of the monocular-auxiliary sensor.; para. 32, ll. 10-12: Obtain at least 3 points on the ground-plane from the descriptive point cloud to estimate the ground-plane's normal in the original coordinate.).
It is unclear if Zhang, as modified, explicitly discloses the motion data is determined further based on vehicle motion data of the vehicle.
However, Natroshvili discloses motion data is determined based on vehicle motion data of a vehicle (Natroshvili; paras. 133-135: FIG. 7 shows an example of grid cells velocity propagation as vehicles and other objects move in the environment. In turn, FIG. 7 illustratively shows the rationale for formula (7). In FIG. 7, diagram 710 represents the movement of a vehicle v. The rectangle 712 represents the vehicle v at a time t, while the rectangle 714 represents the vehicle v at a time t+1. The arrow 716 illustratively shows the velocity of the vehicle v at time t. Lacking any other information on the movements of the vehicle v, it may be assumed that the velocity of vehicle v will stay constant in the future. In FIG. 7, diagram 720 represents the consequences of the assumption displayed in diagram 710 on the estimation of grid cells velocities.).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have determined the velocity of the surrounding detected objects in the control unit of Zhang, as modified, relative to the vehicle's velocity, as disclosed by Natroshvili, to yield the predictable result of accurately determining the approach speed of an object that may collide with the vehicle.
Regarding claim 8, as best understood, Zhang, as modified, discloses assigning the set of 3D points with associated motion data to the free-space estimation of the scene comprises: assigning each 3D point of the set of 3D points to a cell of a plurality of cells in an occupancy grid of the free-space estimation, based on the three-dimensional position of the 3D points (Zhang; col. 37, ll. 39-40: sensory input is used to update an internal 3D map that is used to update an occupancy grid; col. 38, ll. 14-22: FIG. 27A illustrates an example collision map 2700 for an area coverage application in one implementation. Collision map 2700 is generated by planner to determine a “safe” path around an obstacle 2702 that lies at the center cell of the collision map 2700. When finding a path about an obstacle, the planner can build the collision map 2700 with cells representing the cell size in the occupancy grid and plot known obstacles in the collision map.).
Zhang, as modified, does not appear to explicitly disclose assigning, to each cell in the occupancy grid, aggregated motion data based on the motion data associated with the 3D points assigned to the respective cell.
Natroshvili discloses assigning, to each cell in an occupancy grid, aggregated motion data based on the motion data associated with points assigned to the respective cell (Natroshvili; para. 136: The grid cell 732, which corresponds to Cell n in formula (7), has a single occupancy hypothesis, illustratively represented by the circle 742, with a degree of belief of occupancy b732 and a velocity computed by the function vt(Cell 732).).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have assigned velocities to the grid cells of the occupancy grid of Natroshvili, for cells of the multilayer occupancy grid, in the control unit of Zhang, as modified, with the motivation of aggregating velocity data for a plurality of particles thereby reducing the computation required to compute the position of objects around the vehicle (Natroshvili; para. 204).
Supplemental References
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Gummadi et al., in US 2023/0382423, disclose a robot navigation system that uses camera images to generate a free space estimation of a navigation area represented as an occupancy grid.
Danziger et al., in US 2023/0141515, disclose a method of applying optical flow to determine the position of objects relative to a vehicle using a depth map.
Liu et al., in US 2021/0356599, disclose a method for estimating a moving speed of a pedestrian, relative to the ground, detected by an autonomous vehicle.
Yogamani et al., in US 2019/0080604, discloses a driver assistance system that projects a free-space estimate of the environment surrounding a vehicle onto a ground plane and uses optical flow to determine the positions of static and dynamic objects in the environment.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSEPH THOMPSON whose telephone number is (571)272-3660. The examiner can normally be reached Mon-Thurs 9:00AM-3:00PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Erin Bishop can be reached at (571)270-3713. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSEPH THOMPSON/Examiner, Art Unit 3665
/Erin D Bishop/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights