DETAILED ACTION
This Office Action is in response to the Amendment filed on 04/10/2026 and is being filed as a second Non-Final for the reasons presented below.
In the filed response, independent claims 1, 4, and 5 have been amended, with claim 3 being canceled.
Accordingly, Claims 1-2 and 4-5 have been examined and are pending.
Response to Arguments
1. Applicant’s remarks, see pgs. 5-7, filed 04/10/2026, with respect to the prior art rejection(s) of the instant claims under 35 U.S.C. 103 have been fully considered and are persuasive, based on the incorporated allowable subject matter of now canceled claim 3. Therefore, the rejections based on Stein and Smith have been withdrawn. However after carefully reconsidering the art of record, it appears Levi et al. US 2023/0298358 A1 and Yasui et al. US 2022/0319198 A1 (see PTO 892), hereinafter referred to as Levi and Yasui, respectively, can reasonably address the amended features given their broadest reasonable interpretation (BRI). As such, a new ground of rejection can be made in view of these two prior art. For these reasons, this office action is submitted as a Non-Final. Please refer to examiner’s responses below for details.
2. After reconsidering the entire art of record in light of the amendments, the examiner finds that Levi reasonably teaches and/or suggests the limitation “wherein the processor is configured to detect the fallen object included in the image based on the image by using a model obtained by performing learning using teacher data which is a data set of a learning image of the front of a learning vehicle shot by a camera of the learning vehicle and a label showing whether a shadow of a learning fallen object is included in the learning image, wherein at least an area in which the learning fallen object affecting the travel of the learning vehicle and the shadow of the learning fallen object may exist is cut out and used for the learning of the model, the area in which the learning fallen object and the shadow of the learning fallen object may exist being included in the learning image.” as recited for e.g. in Claim 1. In the context of manned and unmanned vehicles, Levi discloses a scene analyzer engine within a shadow-based object detection and identification (ODI) system (see for e.g. ¶0018 and figs. 1 and 16) that can employ one or more machine learning models (e.g. ANNs) for classifying a detected object as an obstacle or non-obstacle (e.g. ¶0044 and ¶0051). Although Levi does not provide details with respect to “learning” (i.e. “by performing learning using teacher data which is a data set of a learning image of the front of a learning vehicle shot by a camera of the learning vehicle”), these features are believed to be integral to machine learning models. Yasui’s teachings are also deemed relevant with respect to the foregoing, since they describe the training image and teacher data for training the parameters of a machine learning model (e.g. DNN) for an in-vehicle device (e.g. ¶0054-¶0055). Levi also does not use the term “label”, however, it is also believed that supervised machine learning techniques rely on labels for classification purposes. Since ¶0051 shows supervised machine learning techniques may be employed for object classification, the use of labels is deemed inherent to the process. With the use of ANNs for implementing Levi’s methods (e.g. shadow detection - e.g. ¶0051), and given the inherent features of machine learning models, which in turn are further disclosed/suggested in Yasui, Levi’s teachings are believed to be relevant, particularly since they can determine based on at least one shadow-related characteristic, whether a scene in an image includes an object, and whether the object is an obstacle or not for the traveling vehicle (e.g. fig. 16). For these reasons, which are further elaborated on in the office action below, the examiner respectfully submits Levi and Yasui, either alone or in combination, reasonably teach and/or suggest the amended features of the instant claims given their BRI.
3. Lastly, after further considering the limitation “acquire an image of the front of a vehicle shot by a camera located in a position out of a lower portion which is lower than a line segment connecting a headlight of the vehicle and a fallen object on a road surface in front of the vehicle, the line segment being included in a vertical plane and the lower portion being included in the vertical plane;” as recited in claim 1 and as similarly recited in claims 4-5, the examiner respectfully submits the term “lower portion” relative to the line segment that connects the vehicle’s headlight with the fallen object is not clearly defined. Figs. 2A-2B of the Instant Disclosure show the camera’s position and the line segments that connect the vehicle’s headlight with the fallen object, however, whether the location of the camera is lower than the line segments as shown cannot be readily determined. To help illustrate this, Stein’s fig. 3B is presented below, where the line segment connecting the headlight of the vehicle with the object has been color-coded blue. Although fig. 3B is a side view and fig. 2B of the instant disclosure is a top view, they are believed to be similar. As such, it is not clear how the position of camera 12 can be lower than the line segment. Please see office action below for details.
[AltContent: connector]
PNG
media_image1.png
200
400
media_image1.png
Greyscale
4. The Examiner is available to discuss the matters of this office action to help move the Instant Application forward. Please refer to the conclusion to this office action regarding scheduling interviews.
5. Accordingly, Claims 1-2 and 4-5 have been examined and are pending.
Claim Rejections - 35 USC § 112
6. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-2 and 4-5 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claim 1, Claim 1 recites “acquire an image of the front of a vehicle shot by a camera located in a position out of a lower portion which is lower than a line segment connecting a headlight of the vehicle and a fallen object on a road surface in front of the vehicle, the line segment being included in a vertical plane and the lower portion being included in the vertical plane;” (emphasis added). After further considering the foregoing, it is not entirely clear what is meant by “a lower portion” as claimed. For e.g., ¶0020 of the filed specification refers to figs. 1, 2A, 2B, and 2C with respect to camera 12. If a camera (12) is located at a lower portion that is lower than a line segment (LRF/LLF) connecting the vehicle’s headlight (11R/11L) and the fallen object on the road (FT), it seems as though camera 12 should be at a lower position than the headlights. Although camera 12 is at a lower position in fig. 6 (i.e. located to the side of the headlights - ¶0052), it is still unclear as to how this can be lower than the line segments LRF/LLF. Based on this understanding, the metes and bounds of the aforementioned features cannot be unequivocally ascertained.
Regarding Claim 2, Claim 2 depends on Claim 1 and therefore includes all of its limitations. For this reason, Claim 2 is also rejected under 35 U.S.C. 112(b).
Regarding Claims 4-5, Claims 4-5 recite limitation similar to Claim 1 above. For the same reasons presented, Claims 4-5 are also rejected under 35 U.S.C. 112(b).
Claim Rejections - 35 USC § 103
7. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2 and 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Stein US 2013/0147957 A1, in view of Levi et al. US 2023/0298358 A1, and in further view of Yasui et al. US 2022/0319198 A1, hereinafter referred to as Stein, Levi, and Yasui, respectively.
Regarding claim 1, Given the broadest reasonable interpretation (BRI), Stein teaches and/or suggests “A surrounding situation recognition device comprising a processor configured to [See figs. 1-2]: acquire an image of the front of a vehicle shot by a camera located in a position out of a lower portion which is lower than a line segment connecting a headlight of the vehicle and a fallen object on a road surface in front of the vehicle [As per the camera and headlight arrangement in figs. 2A-2B of the instant disclosure and given the BRI of the aforementioned features, please refer to figs. 3A-3B of Stein, where the positions of camera 12 and a headlight of a vehicle relative to an object 34 (¶0046-¶0047) are similarly arranged. The claimed “line segment” can be construed as the line connecting the headlight to object 34. Since Stein’s object is not defined as “fallen”, see Yasui below for support], the line segment being included in a vertical plane [Given the BRI of the aforementioned features, one can define a vertical plane through the vehicle in a longitudinal direction that passes through the headlight. As such, the line connecting Stein’s headlight and object 34 (figs. 3a-3b) will be included in said plane] and the lower portion being included in the vertical plane [please refer to figs. 3A-3B of Stein]; and detect the fallen object included in the image by using the fact that a shadow created on the road surface by the fallen object blocking light emitted from the headlight is included in the image [See shadow 32 of object 34 in figs. 3A-3B], the shadow being positioned on the opposite side of the headlight across the fallen object, [Shadow 32 is on an opposite side of the headlight across object 34 as illustrated] wherein the processor is configured to detect the fallen object included in the image based on the image by using a model obtained by performing learning using teacher data which is a data set of a learning image of the front of a learning vehicle shot by a camera of the learning vehicle and a label showing whether a shadow of a learning fallen object is included in the learning image, at least an area in which the learning fallen object affecting the travel of the learning vehicle and the shadow of the learning fallen object may exist is cut out and used for the learning of the model, the area in which the learning fallen object and the shadow of the learning fallen object may exist being included in the learning image.” [Stein does not address the aforementioned features. Please see Levi below for corresponding support. Yasui also provides support for these features] Since Stein is silent with respect to the amended features above, Levi from the same or similar field of endeavor is relied on to teach and/or suggest “wherein the processor [See Levi’s scene analyzer in fig. 1 (e.g. ¶0044) which is a part of ODI system 1000 that is operable to implement shadow-based detection (SBD) for identifying objects as obstacles (¶0048). Also please note the scene analysis engine in ¶0083] is configured to detect the fallen object included in the image [Fig. 16 for e.g. identifies a method using captured images that can determine whether the scene includes an object on a driving surface based on a shadow-related characteristic. Since an object can be an obstacle, this is construed to be fallen object (see e.g. ¶0003 of the filed specification)] based on the image by using a model obtained by performing learning using teacher data [¶0044 shows artificial neural networks (ANNs) may be employed for classifying the object as either an obstacle or a non-obstacle. Machine learning is known to employ teacher data. See Yasui below] which is a data set of a learning image of the front of a learning vehicle shot by a camera of the learning vehicle [Although Levi does not explicitly recite these features, machine learning models are believed to use the data of a learning image, which in the case of Levi, would include captured images for the purposes of training the ANN in the ODI system for classifying objects. Also see Yasui below] and a label showing whether a shadow of a learning fallen object is included in the learning image [The object in Levi can be classified as an obstacle or a non-obstacle based on the shadow detection performed (e.g. ¶0044 and ¶0051). Given that supervised machine learning techniques may be employed (¶0051), this indicates labels are used], at least an area in which the learning fallen object affecting the travel of the learning vehicle and the shadow of the learning fallen object may exist is cut out and used for the learning of the model [¶0051 suggests a corresponding area containing the shadow/object. The term “cut out” is not explicitly recited, however, since this does not appear to be further defined in the filed specification and given its BRI, it is believed Levi’s use of machine learning for performing object classification based on shadow detection (above) would place the shadow/object data in a format suitable for learning of the model. Regarding “cut-out”, please see Yasui below], the area in which the learning fallen object and the shadow of the learning fallen object may exist being included in the learning image.” [See ¶0051 of Levi which suggests the foregoing features] Based on Levi’s shadow-based object detection and identification (ODI) system (see for e.g. ¶0018 and figs. 1 and 16), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the techniques of Stein for analyzing shadows for detecting objects at night (e.g. abstract), to add the scene analyzer engine of Levi as above which can be configured to implement artificial intelligence functionalities to facilitate classifying objects as obstacles or non-obstacles along a driving surface (e.g. ¶0044-¶0045); hence, the safety of a driver can be improved. Although Levi is deemed relevant art, with respect to the use of machine learning models for performing shadow-based object detection and identification, Levi does not explicitly refer to some of the claimed terms, including “teacher data”, etc., although it is believed these terms would be within the level of skill in the art of machine learning. Also not explicitly disclosed in Stein and Levi is the term “fallen object”, although Stein’s objects that are classified as obstacles can be construed as “fallen” based on ¶0003 of the filed specification. As such, the work of Yasui from the same or similar field of endeavor is relied on to further teach and/or suggest “based on the image by using a model obtained by performing learning using teacher data which is a data set of a learning image of the front of a learning vehicle shot by a camera of the learning vehicle” [See for e.g. ¶0053-¶0055 with respect to training processing]. Yasui also explicitly addresses “falling objects” [See for e.g. ¶0048 and ¶0077]. Lastly, regarding the limitation “cut-out” in the above limitation [See for e.g. ¶0075-¶0077 of Yasui with respect to extracting/cutting portions of a captured image which can be subsequently analyzed for determining whether a point of interest is for e.g. a falling object] Based on Yasui’s teachings, it would have therefore been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the obstacle detection in Stein’s driver assistance system (abstract) along with Levi’s shadow-based object detection and identification (ODI) system (fig. 1), to add the teachings of Yasui as above for generating a trained model for discriminating an object present on a road in a captured image from a moving vehicle (e.g. abstract and ¶0004). As such, a range covered by training processing can be expanded (e.g. ¶0048).
Regarding claim 2, Stein, Levi, and Yasui teach all the limitations of claim 1, and are analyzed as previously discussed with respect to that claim. Stein does not address the features of claim 2. Although Levi’s teachings disclose machine learning models in the context of a shadow-based object detection and identification (ODI) system (e.g. fig. 1), the work of Yasui from the same or similar field of endeavor is brought in to teach and/or suggest, “wherein the processor is configured to extract an area in which the fallen object affecting travel of the vehicle and the shadow of the fallen object may exist, the area being included in the image, and detect the fallen object based on the determination result of whether the shadow is included in the area.” [In the context of object detection for a vehicle (e.g. abstract and ¶0069), see ¶0075-¶0077 with respect to extracting/cutting portions of a captured image which can be analyzed for determining whether a point of interest is for e.g. a falling object] The motivation for combining Stein, Levi, and Yasui has been discussed in connection with claim 1, above.
Regarding claim 4, claim 4 is rejected under the same art and evidentiary limitations as determined for the device of claim 1.
Regarding claim 5, claim 5 is rejected under the same art and evidentiary limitations as determined for the device of claim 1. As to the claimed hardware and software, please see for e.g. fig. 1 and ¶0083-¶0087 of Levi for support.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Please see PTO 892 for additional references. Relevant works include Protter et al. US 9,928,441 B2, which employ shadowing analysis (e.g. abstract), O’Cualain et al. US 2015/0291097 A1which can detect shadows of an object (e.g. ¶0021), and Park KR 20220055257 A for performing shadow processing in a camera image.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RICHARD A HANSELL JR. whose telephone number is (571)270-0615. The examiner can normally be reached Mon - Fri 10 am- 7 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jamie Atala can be reached at 571-272-7384. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RICHARD A HANSELL JR./Primary Examiner, Art Unit 2486
Prosecution Insights