Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claims 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-9, 11-16 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over O’ Hara et al., (US12158762) hereafter O’Hara in view of NPL1 (CLIP2: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Data, Yihan Zeng et al., arXiv, 26 mar 2023, Pages 1-15) hereafter NPL1 .
1. Regarding claim 1, O’Hara discloses an apparatus (figs 1, 2A-2C and 3, col 9 lines 32 through col 10 lines 4, col 12 lines 10 through col 17 lines 60 shows and discloses an apparatus) comprising:
at least one memory (fig 1 memory 124, col 9 lines 32 through col 10 lines 4 shows and describes at least one memory); and
at least one processor coupled to the at least one memory, the at least one processor configured (fig 1 processor 122 coupled to memory 124, col 9 lines 32 through col 10 lines 4 shows and describes the at least one processor coupled to the at least one memory configured to process the instructions) to:
receive road data, wherein the road data represents a real-world environment encountered by an autonomous vehicle (AV) and wherein the road data comprises point cloud data representing a plurality of objects (figs 1, 2B-2C,3 (301), col 12 lines 10-34, col 14 lines 65 through col 15 lines 10 shows and discloses a LIDAR image that contains the point cloud (depth information) of captured objects in front of the autonomous vehicle on the road meeting the claim limitations) and;
generate, for each of the plurality of objects, a corresponding set of first embeddings (fig 2C, col 14 lines 21-27 shows and discloses generate by the text encoder 293, for each of the plurality of objects, a corresponding set of first embeddings TE_1, TE_2------ TE_M corresponding to the plurality of the objects A, B----M meeting the claim limitations)
receive a text string corresponding to a searched object (figs 2C col 14 lines 1-27 shows and discloses receiving by the text encoder a text string corresponding to a searched object (i.e for example Text string describing object A, Text string B describing object B—etc.) meeting the above claim limitations);
generate a second embedding corresponding to the searched object (figs 2C, col 14 lines 27-44 shows and discloses generating an image embedding X corresponding to the searched object (for example object A, object B etc.) meeting the claim limitations)
identify a matching object among the plurality of objects based on a comparison of the set of first embeddings and the second embedding (fig 2C, col 14 lines 30-51 discloses identifying a particular object based on the similarity measures between the image embedding X (second embedding) and the text embeddings TE_1----TE_M (first set of the embeddings) and determines that image X matches the particular object (i.e a matching object) meeting the above claim limitations). As seen above O’Hara discloses set of embeddings. O’Hara is silent and however fails to disclose wherein each embedding in the first set of embeddings is a multi-modal embedding including at least text data, image data and point-cloud data
and wherein the second embedding is a multi-modal embedding including at least text data, image data and point-cloud data.
NPL1 discloses wherein each embedding in the first set of embeddings is a multi-modal embedding including at least text data, image data and point-cloud data (fig 2, page 3 shows Text encoder (embedding), Image encoder (embedding) and PC encoder (Point-cloud embedding) in the cross-modal (multi-modal) pretraining (generating) meeting the above claim limitations) and wherein the second embedding is a multi-modal embedding including at least text data, image data and point-cloud data (fig 2, page 3 shows the recognition (i.e the searched object) output data ET (Text data), EP (point cloud data) and EI (Image data) in the CLIP Zero-shot recognition meeting the above claim limitations). Before the effective filing date of the invention was made, O’Hara and NPL1 are combinable because they are from the same filed of endeavor and are analogous art of image processing. The suggestion/motivation would be an optimized, enhanced and increased quality system/method on page 2 col 2 lines 14-27. Therefore, it would be obvious and within one of ordinary skill in the art to have recognized the advantages of NPL1 in the apparatus of O’Hara to obtain the invention as specified in claim 1.
2. Regarding claim 2, O’Hara and NPL1 discloses the apparatus of claim 1. O’Hara discloses further wherein the comparison is based on a set of distances between the second embedding and the set of first embeddings (col 3 lines 28-38 and col 19 lines 13-62 discloses wherein the comparison is based on a set of distances (i.e cosine distances) between the second embedding (i.e the image embedding) and the set of first embeddings (i.e plurality of text embeddings) meeting the claim limitations).
3. Regarding claim 4, O’Hara and NPL1 discloses the apparatus of claim 1. O’Hara discloses further wherein the second embedding is generated using the text string (figs 2A, 2C, col 14 lines 1-51 shows and discloses wherein the second embedding (i.e the image embedding) is generated using the text string meeting the claim limitations).
4. Regarding claim 5, O’Hara and NPL1 discloses the apparatus of claim 1. O’Hara discloses further wherein the at least one processor is further configured to: receive an image corresponding to the searched object (fig 2C shows and discloses wherein the at least one processor is further configured to: receive an image corresponding to the searched object).
5. Regarding claim 6, O’Hara and NPL1 discloses the apparatus of claim 5. O’Hara discloses further wherein the second embedding is based on the image (figs 2C, col 14 lines 27-44 shows and discloses generating an image embedding X corresponding to the searched object (for example object A, object B etc.) meeting the claim limitations).
6. Regarding claim 7, O’Hara and NPL1 discloses the apparatus of claim 1. O’Hara discloses further wherein for each object of the plurality of objects, embedding in the corresponding set of first embeddings comprises a vector representing characteristics of the object in the plurality of the objects (fig 2C and col 11 lines 1-11 discloses wherein for each object of the plurality of objects, embedding in the corresponding set of first embeddings comprises a vector representing characteristics of the object in the plurality of the objects).
7. Claim 8 is a corresponding method claim of claim 1. See the corresponding explanation of claim 1.
8. Claim 9 is a corresponding method claim of claim 2. See the corresponding explanation of claim 2.
9. Claim 11 is a corresponding method claim of claim 4. See the corresponding explanation of claim 4.
10. Claim 12 is a corresponding method claim of claim 5. See the corresponding explanation of claim 5.
11. Claim 13 is a corresponding method claim of claim 6. See the corresponding explanation of claim 6.
12. Claim 14 is a corresponding method claim of claim 7. See the corresponding explanation of claim 7.
13. Claim 15 is a corresponding non-transitory computer readable medium claim of claim 1. See the corresponding explanation of claim 1.
14. Claim 16 is a corresponding non-transitory computer readable medium claim of claim 2. See the corresponding explanation of claim 2.
15. Claim 18 is a corresponding non-transitory computer readable medium claim of claim 4. See the corresponding explanation of claim 4.
16. Claim 19 is a corresponding non-transitory computer readable medium claim of claim 5. See the corresponding explanation of claim 5.
17. Claim 20 is a corresponding non-transitory computer readable medium claim of claim 6. See the corresponding explanation of claim 6.
Claims 3, 10 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over O’ Hara in view of NPL1 and in further view of CHEN et al.(US20230047160) hereafter CHEN.
18. Regarding claim 3, O’Hara and NPL1 discloses the apparatus of claim 2. O’Hara discloses further wherein the matching object is a lowest (col 3 lines 28-38, col 11 lines 31-56 and col 19 lines 13-62 discloses wherein the matching object is the lowest cosine distance in the set of distances). O’Hara discloses other similarity distance measures as seen in the disclosure. O’Hara and NPL1 are however silent and fails to disclose a lowest Euclidean distance.
CHEN discloses a lowest Euclidean distance of the matching object in para 0064. Before the effective filing date of the invention was made, CHEN, O’Hara and NPL1 are combinable because they are from the same filed of endeavor and are analogous art of vehicle data processing. The suggestion/motivation would be an efficient system (i.e minimizes aggregate loss) at para 0016. Therefore, it would be obvious and within one of ordinary skill in the art to have recognized the advantages of CHEN in the apparatus of O’Hara and NPL1 to obtain the invention as specified in claim 3.
19. Claim 10 is a corresponding method claim of claim 3. See the corresponding explanation of claim 3.
20. Claim 17 is a corresponding non-transitory computer readable medium claim of claim 3. See the corresponding explanation of claim 3.
Examiner's Note: Examiner has cited figures, and paragraphs in the references as applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested for the applicant, in preparing the responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. Examiner has also cited references in PTO892 but not relied on, which are relevant and pertinent to the applicant’s disclosure, and may also be reading (anticipatory/obvious) on the claims and claimed limitations. Applicant is advised to consider the references in preparing the response/amendments in-order to expedite the prosecution.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAYESH PATEL whose telephone number is (571)270-1227. The examiner can normally be reached IFW Mon-FRI.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at 571-270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JAYESH PATEL
Primary Examiner
Art Unit 2677
/JAYESH A PATEL/Primary Examiner, Art Unit 2677
Prosecution Insights