DETAILED ACTION
This is a response to the Amendment to Application # 17/885,300 filed on September 29, 2025 in which claims 1, 4, 10, 13, 19, and 105 were amended.
Continued Examination Under 37 C.F.R. § 1.114
A request for continued examination under 37 C.F.R. § 1.114, including the fee set forth in 37 C.F.R. § 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 C.F.R. § 1.114, and the fee set forth in 37 C.F.R. § 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 C.F.R. § 1.114. Applicant's submission filed on September 29, 2025 has been entered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1, 4-10, 13-19, and 105-108 are pending, of which claims are rejected under 35 U.S.C. § 103.
Claim Rejections - 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims, the Examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicants are advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 1, 4, 6-10, 13, 15-19, and 105-108 are rejected under 35 U.S.C. § 103 as being unpatentable over Ben Shalom et al., US Publication 2015/0210276 (hereinafter Ben Shalom), as cited on Notice of References Cited dated July 15, 2024, in view of Sajjadi Mohammadabadi et al., US Publication 2020/0293796 (hereinafter Sajjadi Mohammadabadi).
Regarding claim 1, Ben Shalom discloses a system for navigating a vehicle, the system comprising “at least one processor comprising circuitry and a memory.” (Ben Shalom ¶ 85). Additionally, Ben Shalom discloses “wherein the memory includes instructions that when executed by the circuitry cause the at least one processor to: receive a first image frame acquired by an image capture device” (Ben Shalom ¶ 127) where the forward facing cameras (i.e., image capture devices) may receive images. Further, Ben Shalom discloses “wherein the first image frame is representative of an environment of the vehicle, the environment including at least one traffic light” (Ben Shalom ¶ 79) where the images are representative of a traffic light above the vehicle (i.e., in an environment of the vehicle). Moreover, Ben Shalom discloses “detect in the first image frame a representation of the at least one traffic light” (Ben Shalom ¶ 315) where traffic lights are determined from a set of images and includes determining a color associated with the traffic light. Likewise, Ben Shalom discloses “receive a second frame and a third acquired by the image capture device, wherein the second image frame and the third image frame include representations of the at least one traffic light” (Ben Shalom ¶ 315) by analyzing “a series of acquired images” (i.e., second and third image frames) of the traffic light. Ben Shalom also discloses “determine motion information indicating a motion of the vehicle between a time when the first image frame was acquired and subsequent times when the second image frame and the third image frame were acquired, the motion information including at least a distance traveled by the host vehicle between the time when the first image frame was acquired and the subsequent times” (Ben Shalom ¶ 284) by detecting that the vehicle has traveled 10 meters between the captured images frames.
In addition, Ben Shalom discloses “determine, based on the motion information and a location in the first image frame of the representation of the at least one traffic light, expected locations of the representations of the at least one traffic light in the first second image frame and the third image frame” (Ben Shalom ¶ 155) by “track[ing] the candidate objects,” which is determining a position of the object in each analyzed frame (including the at least one additional image frame), and using the data such as position relative to vehicle 200 (i.e., the motion information) and a “location[] in the images likely to contain traffic lights.” Ben Shalom discloses that the candidate objects may be traffic lights. (Ben Shalom ¶ 167).
Furthermore, Ben Shalom discloses “extract subsections of the second image frame and the third image frame based on expected locations” (Ben Shalom ¶ 314) by analyzing only the locations representative of the individual traffics lights, those locations must necessarily have been selected (i.e., extracted1) from the remainder of the image. This is based on their expected locations or the system would not know what locations to extract. Moreover, Ben Shalom discloses “determine, based on a comparison of at least a portion of the first image frame and the extracted subsections, that the at least one traffic light includes a blinking lamp” (Ben Shalom ¶ 315) by determining that the traffic light is flashing based on comparing the images to determine that a first image is “illuminated” and a second image is “darkened.” The images include the detected traffic lights and, thus, include the expected locations of those traffic lights. Likewise, Ben Shalom discloses “determine a color state associated with the blinking lamp” (Ben Shalom ¶ 315) by determining a color associated with the lamp. Finally, Ben Shalom discloses “based on the determination that the at least one traffic light includes the a blinking lamp and the color state of the blinking lamp, cause the vehicle to implement a navigational action relative the at least one traffic light” (Ben Shalom ¶ 316) by causing a system response based on the status of the traffic light, which includes both the color and blinking status.
Ben Shalom does not appear to explicitly disclose that “the expected locations being determined based on expected changes in image coordinates corresponding to the distance traveled by the host vehicle.”
However, Sajjadi Mohammadabadi discloses a method for detecting objections at an intersection, including traffic lights (Sajjadi Mohammadabadi ¶ 38), that predicts pixels corresponding to bounding boxes (i.e., expected locations) of the items, wherein “the expected locations being determined based on expected changes in image coordinates corresponding to the distance traveled by the host vehicle” (Sajjadi Mohammadabadi ¶ 77) where the predictions are adjusted (i.e., the expected locations are determined) based on prior predictions (i.e., using expected changes), the prior predictions being bounding box coordinates 108, using (i.e., corresponding to) motion information between consecutive images (i.e., the distance traveled by the host vehicle).
Ben Shalom and Sajjadi Mohammadabadi are analogous art because they are from the “same field of endeavor,” namely that of traffic light detection systems.
Prior to the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Ben Shalom and Sajjadi Mohammadabadi before him or her to modify the general action of detecting an expected position of the traffic light of Ben Shalom to include the specific method for detecting an expected position of the traffic light of Sajjadi Mohammadabadi.
The motivation/rationale for doing so would have been that of applying a known technique to a known device. See KSR Int’l Co. v. Teleflex Inc., 550 US 398, 82 USPQ2d 1385, 1396 (U.S. 2007) and MPEP § 2143(I)(D). Ben Shalom teaches the “base device” for detecting a traffic light based on an expected location. Further, Sajjadi Mohammadabadi teaches the “known technique” for detecting a traffic light based on an expected location where the expected location is “determined based on expected changes in image coordinates corresponding to the distance traveled by the host vehicle” that is applicable to the base device of Ben Shalom. One of ordinary skill in the art would have recognized that applying the known technique would have yielded predictable results and resulted in an improved system.
Regarding claim 10, it merely recites a method implemented by the system of claim 1. The method comprises computer software modules for performing the various functions. The combination of Ben Shalom and Sajjadi Mohammadabadi comprises computer software modules for performing the same functions. Thus, claim 10 is rejected using the same rationale set forth in the above rejection for claim 1.
Regarding claim 19, it merely recites a non-transitory computer readable medium embodying the system of claim 1. The medium comprises computer software modules for performing the various functions. The combination of Ben Shalom and Sajjadi Mohammadabadi comprises computer software modules for performing the same functions. Thus, claim 19 is rejected using the same rationale set forth in the above rejection for claim 1.
Regarding claims 4, 13, and 105, the combination of Ben Shalom and Sajjadi Mohammadabadi discloses the limitations contained in parent claims 1, 10, and 19 for the reasons discussed above. In addition, the combination of Ben Shalom and Sajjadi Mohammadabadi discloses “wherein the comparison includes providing the at least a portion of the first image frame and the at least a portion of the at least one additional image frame to a long short term memory (LSTM) network” (Sajjadi Mohammadabadi ¶ 55).
Regarding claims 6, 15, and 106, the combination of Ben Shalom and Sajjadi Mohammadabadi discloses the limitations contained in parent claims 1, 10, and 19 for the reasons discussed above. In addition, the combination of Ben Shalom and Sajjadi Mohammadabadi discloses “wherein the navigational action includes stopping of the vehicle in response to a determination that the at least one traffic light includes a blinking lamp having a red color state” (Ben Shalom ¶ 315) where the light may be recognized to be flashing (i.e., blinking) and have any of a color red, yellow, or green.
Regarding claims 7, 16, and 107, the combination of Ben Shalom and Sajjadi Mohammadabadi discloses the limitations contained in parent claims 1, 10, and 19 for the reasons discussed above. In addition, the combination of Ben Shalom and Sajjadi Mohammadabadi discloses “wherein the navigational action includes slowing of the vehicle in response to a determination that the at least one traffic light includes a blinking lamp having a yellow color state” (Ben Shalom ¶ 315) where the light may be recognized to be flashing (i.e., blinking) and have any of a color red, yellow, or green.
Regarding claims 8 and 17, the combination of Ben Shalom and Sajjadi Mohammadabadi discloses the limitations contained in parent claims 1 and 10 for the reasons discussed above. In addition, the combination of Ben Shalom and Sajjadi Mohammadabadi discloses “wherein the navigational action includes causing the vehicle to yield in response to a determination that the at least one traffic light includes a blinking lamp having a yellow color state and wherein the blinking lamp includes a directional arrow” (Ben Shalom ¶¶ 315-316) where the light may be recognized to be flashing (i.e., blinking) and having a directional arrow.
Regarding claims 9, 18, and 108, the combination of Ben Shalom and Sajjadi Mohammadabadi discloses the limitations contained in parent claims 1, 10, and 19 for the reasons discussed above. In addition, the combination of Ben Shalom and Sajjadi Mohammadabadi discloses “wherein the navigational action includes causing the vehicle to yield in response to a determination that the at least one traffic light includes a blinking lamp having a green color state” (Ben Shalom ¶ 315) where the light may be recognized to be flashing (i.e., blinking) and have any of a color red, yellow, or green.
Claims 5 and 14 are rejected under 35 U.S.C. § 103 as being unpatentable over Ben Shalom in view of Sajjadi Mohammadabadi, as applied to claims 4 and 13 above, and in further view of Steven Parker; What binary number represents off?; January 6, 2016; Treehouse Community; Pages 1-2, as cited on the Notice of References Cited dated July 15, 2025.
Regarding claims 5 and 14, the combination of Ben Shalom and Sajjadi Mohammadabadi discloses the limitations contained in parent claims 4 and 13 for the reasons discussed above. In addition, the combination of Ben Shalom and Sajjadi Mohammadabadi does not appear to explicitly disclose “wherein the LSTM network is configured to output for each of the one or more lamps included in the at least one traffic light a logical "0" if the no blinking is detected and a logical "1" if blinking is detected.”
However, Parker discloses that the generally accepted convention is that 0 represents “off” while 1 represents “on.” (Parker 2). A person of ordinary skill in the art would have recognized that when Parker was combined with Ben Shalom and Sajjadi Mohammadabadi, that no blinking would be represented by 0 because blinking is “off,” while blinking would be represented by 1 because blinking is “on.” Therefore the combination of Ben Shalom, Sajjadi Mohammadabadi, and Parker at least teaches and/or suggests the claimed limitation “wherein the LSTM network is configured to output for each of the one or more lamps included in the at least one traffic light a logical "0" if the no blinking is detected and a logical "1" if blinking is detected,” rendering it obvious.
Ben Shalom, Sajjadi Mohammadabadi, and Parker are analogous art because they are from the “same field of endeavor,” namely that of computer programming.
At the time of the invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Ben Shalom and Sajjadi Mohammadabadi and Parker before him or her to modify the assignment of values of the LSTM of Ben Shalom and Sajjadi Mohammadabadi to include the use of 0 for “off” and 1 for “on” of Parker.
The motivation for doing so would have been that such an assignment follows the generally accepted convention of programmers. (Parker 2).
Response to Arguments
Applicant’s arguments filed September 29, 2025, with respect to the rejection of claims 1, 4-10, 13-19, and 105-108 under 35 U.S.C. §§ 102 and 103 (Remarks 10-14) have been considered but are moot in view of the new grounds of rejection.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure:
Kang et al., US Publication 2021/0174516, System and method for extracting traffic light data from changes in coordinates based on motion of the vehicle.
Liu et al., US Patent 10,794,710, System and method for extracting traffic light data from changes in coordinates based on motion of the vehicle.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW R DYER whose telephone number is (571)270-3790. The examiner can normally be reached Monday-Thursday 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aniss Chad can be reached on 571-270-3832. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW R DYER/Primary Examiner, Art Unit 3662
1 “Extract.” Merriam-Webster.com Dictionary, Merriam-Webster, www.merriam-webster.com/dictionary/extract. Accessed 6 Feb. 2020, Page 2.