Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 and 9 have been considered but are moot in view of the new grounds of rejection.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-3, 7-11, 15, and 16 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wang et al. (Wang) (US 2021/0374547).
Regarding claim 1, Wang discloses a method for image processing, comprising:
receiving a plurality of image frames, wherein a traffic sign is depicted in the plurality of image frames ([0067], a video (i.e. a plurality of frames) is input, [0197], a traffic sign in an image is detected over multiple frames);
extracting image features corresponding to the traffic sign based on the plurality of image frames ([0144], [0197], object and text recognition is performed using extracted features from a traffic sign);
determining, using at least one machine learning model, an embedding based on the image features and text included on the traffic sign ([0095], [0099], text-image embedding network); and
determining, using the at least one machine learning model, at least one natural language descriptor of the traffic sign based on the embedding ([0055], natural language processing is used for classifying and annotating images; [0082], [0619], [0657], labels for images are generated using natural language processing), wherein the at least one natural language descriptor includes at least one characteristic of the traffic sign other than the text included in the traffic sign ([0197], a first neural network identifies the sign as a traffic sign (i.e. a first characteristic) while a second neural network interprets text written on a sign (i.e. a second characteristic) and a third neural network identifies flashing lights on a sign (i.e. a third characteristic).
Regarding claims 2 and 10, Wang discloses wherein the at least one natural language descriptor is determined using a large language machine learning model of the at least one machine learning model ([0094], large scale data set with natural language processing (NLP) generated image labels; [0103], a large data set is used).
Regarding claims 3 and 11, Wang discloses wherein at least one characteristic of the traffic sign is in the group consisting of: a shape, a color, a type, and an intended recipient ([0197], a first neural network identifies the sign as traffic sign (i.e. a type)).
Regarding claims 7 and 15, Wang discloses wherein the plurality of image frames are received from at least one camera (FIG. 9B, cameras 970-998) disposed on a vehicle (900).
Regarding claims 8 and 16, Wang discloses wherein the operations further include controlling a function of a vehicle based on the at least one natural language descriptor ([0142], [0196], natural language descriptors generated from reading traffic signs are used for autonomous route planning, navigation, and braking).
Regarding claim 9, Wang discloses an apparatus, comprising:
a memory storing processor-readable code ([0086], a stored program is executed by a processor); and
at least one processor coupled to the memory, the at least one processor configured to execute the processor-readable code to cause the at least one processor to perform operations ([0086], a stored program is executed by a processor) including:
receiving a plurality of image frames, wherein a traffic sign is depicted in the plurality of image frames ([0067], a video (i.e. a plurality of frames) is input, [0197], a traffic sign in an image is detected);
determining, using at least one machine learning model, image features corresponding to the traffic sign based on the plurality of image frames ([0066], image feature detection using machine learning; [0144], [0197], object and text recognition is performed using extracted features from a traffic sign);
determining, using the at least one machine learning model, text features associated with the traffic sign based on the plurality of image frames ([0144], [0197], object and text recognition is performed using extracted features from a traffic sign);
determining, using the at least one machine learning model, an embedding that combines the image features and the text features ([0067], [0082], [0196], text of the traffic sign is interpreted); and
determining, using the at least one machine learning model, at least one natural language descriptor of the traffic sign based on the embedding ([0055], natural language processing is used for classifying and annotating images; [0082], [0619], [0657], labels for images are generated using natural language processing), wherein the at least one natural language descriptor includes at least one characteristic of the traffic sign other than the text included in the traffic sign ([0197], a first neural network identifies the sign as a traffic sign (i.e. a first characteristic) while a second neural network interprets text written on a sign (i.e. a second characteristic) and a third neural network identifies flashing lights on a sign (i.e. a third characteristic).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 4, 6, 12, and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (Wang) (US 2021/0374547) in view of Chen et al. (Chen) (US 2023/0343109).
Regarding claims 4 and 12, Wang discloses the apparatus of claim 9 (see claim 9 above).
Wang is silent about wherein the plurality of image frames also depict a plurality of traffic lights, the operations further including determining a relevant traffic light of the plurality of traffic lights based on a state of at least one of the plurality of traffic lights, at least one detected traffic lane, and a trajectory of at least one detected vehicle.
Chen from the same or similar field of endeavor discloses wherein the plurality of image frames also depict a plurality of traffic lights ([0014], multiple traffic lights are imaged), the operations further including determining a relevant traffic light of the plurality of traffic lights based on a state of at least one of the plurality of traffic lights ([0015], [0031], the color of the traffic light is determined), at least one detected traffic lane ([0023], the traveling lane of the vehicle and an orientation of the vehicle relative to the traffic light is determined), and a trajectory of at least one detected vehicle ([0023], the traveling lane of the vehicle and an orientation of the vehicle relative to the traffic light is determined).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Chen into the teachings of Wang for more accurately controlling a vehicle based only on relevant traffic information.
Regarding claims 6 and 14, Chen further discloses wherein determining the relevant traffic light includes: determining at least one association between the plurality of traffic lights and the at least one detected traffic lane ([0023], [0040], the traveling lane of the vehicle and an orientation of the vehicle relative to the traffic lights is determined); and determining at least one association between the plurality of traffic lights and the at least one detected vehicle ([0023], [0040], the traveling lane of the vehicle and an orientation of the vehicle relative to the traffic lights is determined).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Chen into the teachings of Wang for more accurately controlling a vehicle based only on relevant traffic information.
Claim(s) 5 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (Wang) (US 2021/0374547) in view of Chen et al. (Chen) (US 2023/0343109), and further in view of Eldar et al. (Eldar) (US 2021/0064057).
Regarding claims 5 and 13, Wang in view of Chen discloses the apparatus of claim 12 (see claim 12 above).
Wang in view of Chen is silent about wherein determining the relevant traffic light includes determining a graph-based data structure representative of an intersection, the at least one detected traffic lane, and the at least one detected vehicle, wherein the intersection includes the plurality of traffic lights.
Eldar from the same or similar field of endeavor discloses wherein determining the relevant traffic light ([0473], a relevant traffic light is determined) includes determining a graph-based data structure (FIG. 37A) representative of an intersection (FIG. 36A), the at least one detected traffic lane (FIG. 37A, 3611A), and the at least one detected vehicle (3601) ([0503], FIG. 37A illustrates a possible relation between the time-dependent navigational information of an autonomous vehicle (e.g., vehicle 3601) traveling in lane 3611A as shown in FIG. 36A, and time-dependent state identifier for a traffic light), wherein the intersection includes the plurality of traffic lights (FIG. 36A, [0487], lights 3630A-3630C).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Eldar into the teachings of Wang in view of Chen for more accurately controlling a vehicle based only on relevant traffic information.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEFFERY A WILLIAMS whose telephone number is (571)270-7579. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath Perungavoor can be reached at 571-272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEFFERY A WILLIAMS/ Primary Examiner, Art Unit 2488
Prosecution Insights