DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicants Amendments filed on March 16, 2026 has been entered and made of record.
Currently pending Claim(s): 1-20
Independent Claim(s): 1, 12, 17
Amended Claim(s): 2
Claim Objections
In view of the Applicant’s amendment, the objection to Claim 2 is withdrawn.
Response to Arguments
This office action is responsive to the Applicant’s Arguments/Remarks Made in an Amendment
received on March 16, 2026.
Claim Rejections – 35 USC § 101
Applicant’s arguments, see pg. 8, paragraph 3, and pg. 9, paragraph 1, with respect to Claims 1-5 and 12-13 have been fully considered and are persuasive. The 101 rejection of Claims 1-5 and 12-13 has been withdrawn.
Claim Rejections – 35 USC 103
In the Non-Final Office Action dated, December 8, 2025, the Examiner rejected Claims 1 and 12 under Ham (US PGPub No 2022/0358800 ). The Examiner rejected Claims 17-20 under Ham, Pertsel (US Pat No 12304475), and Kim (US Pat No 11017620). The Applicant disagreed with the rejection, and specifically argued the use of Ham and Kim for the rejections of Claim 1 and 17, respectively .
The Applicant explained (on page 9, paragraph 2 of Applicant’s Argument’s/Remarks) that Ham fails to teach the limitation of recording a video in response to determining that the vehicle is within a threshold distance. The Applicant explained that the first embodiment of Ham teaches that recording is triggered by impact sensor readings, and that the Examiner improperly combined the first and second embodiments of Ham. The Applicant then further argued that the second embodiment teaches recording a video only when is within another vehicle is detected within a threshold distance and is moving over a speed limit. Finally, the applicant argued that Ham’s speed estimation relies on video analysis, not image analysis.
The Examiner respectfully disagrees with these arguments. The Examiner agrees that the first embodiment of Ham does not teach the limitations of Claim 1. The first embodiment of Ham is configured to record a video in response to detecting an impact reading greater than a threshold value (see paragraph [0069). However, the Examiner specifically cited the second embodiment of Ham, which teaches detecting a vehicle (see paragraph [0073], determining that a portion of the vehicle is within a threshold distance (see paragraph [0072], and determining that the vehicle within the threshold distance of the apparatus is moving (see paragraph [0073]). In response to detecting this situation, a video is recorded (see paragraph [0072], “In the present connection, the controller 70 may store a drive video filmed by the camera 30 in the storage 10”). The Examiner initially cited paragraph [0009] which states, “a controller that detects a dangerous situation of the vehicle based on the rear-side video filmed by the camera, and records the front video, the rear video, and the rear-side video of the vehicle filmed by the camera as an event video in storage in the detected dangerous situation”. However, it is clear from paragraph [0072] that the second embodiment of Ham is capable of recording a video in response to detecting a car through image analysis, not solely through recording pressure sensor readings.
As to the second argument, the Examiner argues that Ham teaches the limitation of ‘the vehicle is moving’. Ham teaches that recording begins when a vehicle is detected within a reference distance, and is higher than a speed limit (see paragraph [0072], “As shown in FIG. 3B, the controller 70 provided in the device configured for recording the drive video of the vehicle according to various exemplary embodiments of the present invention may determine that the dangerous situation has occurred to the host vehicle 200 in a case in which a speed of the rear-side vehicle 300 located within the reference distance (e.g., 5 m) from the host vehicle 200 exceeds the limit speed of the road, in a case in which the speed of the rear-side vehicle 300 is higher than a speed of the host vehicle 200 by a threshold (e.g., 30 km/h) or more”). In order to determine a vehicle is moving above a speed limit, the speed of the vehicle must be determined, and thus the vehicle is inherently determined be moving. Thus, the Examiner argues that the Applicant’s arguments are not commensurate with the broadest reasonable interpretation of Claim 1.
Finally, the claim language is directed towards a ‘detection of a vehicle in at least one image from at least one of the plurality of cameras’. It is well known to those of ordinary skill in the art that video is comprised of multiple sequential image frames. Thus, under the broadest reasonable interpretation, detecting a vehicle from video data meets the limitation of detecting a vehicle in at least one image.
For all of the above reasons, the Examiner maintains the Ham and the rejections of Claims 1-16.
As to Claim 17, Applicant argued (on page 11, paragraph 4 of Applicant’s Arguments/Remarks), that Kim fails to teach the limitation of detecting motion while the motion sensors are in a sleep state. The Applicant explained that Kim teaches the opposite of the claimed approach because Kim’s system, ‘requires other sensors such as shock, capacitance, or motion sensors to detect anomalies’. The Applicant also argued that Kim teaches the opposite of the claimed approach because Kim teaches turning of motion sensors when a vehicle is parked in crowded areas.
The Examiner respectfully disagrees with these arguments. Kim teaches in Col. 2, lines 67-68, and Col.3, lines “Each sensor bundles 10(a), 10(b), 10(c) . . . 10(n) include shock, capacitance, motion, and optical sensors to monitor different anomalies proximate the vehicle”. The optical sensor is part of the sensor bundle, and thus the optical sensor may be used to monitor anomalies. Thus, the Examiner disagrees that Kim teaches that other sensors are ‘required’ to detect anomalies. Furthermore, the Examiner has used Kim merely to teach motion sensors in a sleep state while an optical sensor or camera is active. The reference Ham teaches capturing a video responsive to determining that a moving object is within a threshold (where video is known to be a plurality of sequential images). Pertsel teaches that cameras in a parked car can be configured to capturing images. It is the combination of Ham, Pertsel, and Kim that the Examiner has used to reject the claimed approach disclosed in Claim 17, not solely Kim. It would have been obvious to combine the teachings of Pertsel and Kim with the invention of Ham in order to obtain the invention as claimed in Claim 17.
Thus, for all the reasons above, the Examiner maintains Kim, and the rejection of Claims 17-20.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Ham (US PGPub No 2022/0358800), hereinafter Ham.
As to Claim 1, Ham teaches an apparatus (see paragraph [0007], “Various aspects of the present invention are directed to providing a device and a method for recording a drive video”), comprising:
a plurality of cameras (see paragraph [0053], “The camera 30, which is a module mounted on the vehicle to film a video of a region around the vehicle, may include a front camera 31, a rear camera 32, a first rear-side camera 33, and a second rear-side camera 34”, and see Fig 2B),
each camera having a field of view that includes a respective portion of an external environment of the apparatus (see Fib 2B, where a field of view of each camera is shown);
and processing circuitry (see controller 70 in Fig. 1),
configured to initiate recording of a video of the external environment using the plurality of cameras (see paragraph [0009], “a device configured for recording a drive video of a vehicle includes a camera configured for filming a front video, a rear video, and a rear-side video of the vehicle, and a controller that detects a dangerous situation of the vehicle based on the rear-side video filmed by the camera, and records the front video, the rear video, and the rear-side video of the vehicle filmed by the camera as an event video in storage in the detected dangerous situation”, and see paragraph [0072], “In the present connection, the controller 70 may store a drive video filmed by the camera 30 in the storage 10”), based on
detection of a vehicle in at least one image from at least one of the plurality of cameras (see paragraph [0073], “That is, the controller 70 may detect the host vehicle 200 and the rear-side vehicle 300 as objects in the first rear-side video or in the second rear-side video”, where it is known to one of ordinary skill in the art that video contains multiple sequential image frames).
a determination that at least a portion of the vehicle is within a threshold distance of the apparatus (see paragraph [0072], “As shown in FIG. 3B, the controller 70 provided in the device configured for recording the drive video of the vehicle according to various exemplary embodiments of the present invention may determine that the dangerous situation has occurred to the host vehicle 200 in a case in which a speed of the rear-side vehicle 300 located within the reference distance (e.g., 5 m) from the host vehicle”),
and a determination that the vehicle that is within the threshold distance of the apparatus is moving (see paragraph [0073] That is, the controller 70 may detect the host vehicle 200 and the rear-side vehicle 300 as objects in the first rear-side video or in the second rear-side video, detect a relative speed based on a change in a separation distance for each time between the host vehicle 200 and the rear-side vehicle 300, and estimate the speed of the rear-side vehicle 300 based on the detected relative speed and the collected speed of the host vehicle”, where detecting a speed would indicate the movement of the vehicle).
It is recognized that the citations and evidence provided above are derived from potentially different embodiments of a single reference. Nevertheless, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to employ combinations and sub-combinations of these complementary embodiments, because Ham et al. explicitly motivates doing so at least in paragraphs [101] including “The exemplary embodiments were chosen and described to explain certain principles of the present invention and their practical application, to enable others skilled in the art to make and utilize various exemplary embodiments of the present invention, as well as various alternatives and modifications thereof” and otherwise motivating the combination of using multiple cameras to record a potential crash with the method of determining if a vehicle is in a threshold distance in order to provide the driver with multiple views of the potential crash.
As to Claim 12, Claim 12 teaches a method that performs the same process as the apparatus disclosed in Claim 1. Therefore, the rejection and rationale are analogous to that made in Claim 1.
Claims 2-4 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Ham (US PGPub No 20220358800), hereinafter Ham, in view of Pertsel et al. (US Patent No 12304475), hereinafter Pertsel.
As to Claim 2, Ham teaches an embodiment where that the plurality of cameras and the processing circuitry can be implemented in another vehicle that is parked. However, in this embodiment, recording can only occur when an accident has already occurred (see paragraph [0013], “In various exemplary embodiments of the present invention, the controller may activate the sensor while the vehicle is being parked, and determine that the dangerous situation has occurred to the vehicle to activate the camera, and record the front video, the rear video, and the rear-side video of the vehicle filmed by the camera, in the storage as the event video when the controller determines that the amount of the impact detected by the sensor exceeds a reference value”).
However, Pertsel teaches a vehicular recording apparatus implemented in a parked car (see Fig.3, where a parked vehicle is shown, with several cameras), which is configured to record the external environment to capture a hit-and-run accident (see Col. 36, lines 29-40, “The processors 106a-106n may be configured to record and/or annotate the video frames. The processors 106a-106n may be configured to select a subset of the video frames (e.g., the video frames that correspond to the event). The annotations 494 may be added to the subset of the video frames. The subset of the video frames with the annotations 494 may be stored in the memory 108. The communication devices 110 may be configured to send the video and/or the annotations to the driver 202 (e.g., a text message alert sent to a smartphone ). For example, the driver 202 may be inside a store shopping and receive the annotated video frame 480 that shows the damage 484 and who caused the damage 484.” Since the driver of the vehicle was away from the vehicle at the time of the accident, this accident would be considered a hit-and-run). Pertsel and Ham are combinable because both are from analogous fields of vehicle cameras and accident prevention. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Ham with Pertsel. The motivation for doing so would be to create records of potential accidents when the driver is away from the car. Pertsel teaches in Col. 1, lines 22-39, “As a result, a significant amount of parking accidents occur in parking lots. Often, the accidents happen when one of the cars is unoccupied and parked. The vehicle that gets hit is helpless. The owner of the vehicle that 25 is hit does not even witness the damage until after the collision occurs...It would be desirable to implement automatic collision detection, warning, avoidance and prevention in parked cars.” Thus, it would have been obvious to combine the teachings of Ham with the teachings of Pertsel in order to obtain the invention as taught in Claim 2.
As to Claim 3, Ham fails to teach that the processing circuitry is configured to detect the vehicle in the at least one image by: providing the at least one image to a neural network that has been trained to detect at least vehicles in images; and detecting the vehicle based on an output of the neural network.
However, Pertsel teaches that images can be provided to a neural network, (see Fig. 4, images provided to a CNN (convolutional neural network)), and that a vehicle and other objects can be detected based on the output of the network (see Col. 15, lines 16-22, "The CNN module 150 may determine a likelihood that pixels in the video frames belong to a particular object and/or objects in response to the descriptors. For example, using the descriptors, the CNN module 150 may determine a likelihood that pixels correspond to a particular object (e.g., a person, a vehicle, a car seat, a tree, etc.) and/or characteristics of the object (e.g., a mouth of a person, a hand of a person, headlights of a vehicle, a branch of a tree, a seatbelt of a seat, etc.)”, and see Fig. 4 item 310, where a vehicle is identified by the dashed box). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Ham with Pertsel. The motivation for doing so would be to identify potential vehicles that may cause harm to a parked vehicle. Thus, it would have been obvious to combine the teachings of Ham with the teachings of Pertsel in order to obtain the invention as taught in Claim 3.
As to Claim 4, Ham fails to teach wherein detecting the vehicle comprises determining that the vehicle is not a person. However, Pertsel teaches a neural network that is capable of identifying vehicles and persons (see Col. 15, lines 16-22, "The CNN module 150 may determine a likelihood that pixels in the video frames belong to a particular object and/or objects in response to the descriptors. For example, using the descriptors, the CNN module 150 may determine a likelihood that pixels correspond to a particular object (e.g., a person, a vehicle, a car seat, a tree, etc.) and/or characteristics of the object (e.g., a mouth of a person, a hand of a person, headlights of a vehicle, a branch of a tree, a seatbelt of a seat, etc.)). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Ham with Pertsel. The motivation for doing so would be to identify potential vehicles that may cause harm to a parked vehicle. Thus, it would have been obvious to combine the teachings of Ham with the teachings of Pertsel in order to obtain the invention as taught in Claim 4.
As to Claim 13, Claim 13 teaches the same limitation as Claim 3, and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are analogous to that made in Claim 3.
As to Claim 14, Ham fails to teach transforming a location in the at least one image that corresponds to the at least the portion of the vehicle into a physical location in the external environment using one or more known parameters of the at least one of the plurality of cameras.
However, Pertsel teaches that camera parameters can be used to calculate distances (see Col. 15, lines 18-39, “The disparity engine 164 may be configured to determine a distance based on images captured as a stereo pair. Two or more of the capture devices 102 a-102 n may be configured as a stereo pair of cameras. The capture devices 102 a-102 n configured as a stereo pair may be implemented close to each other at a pre-defined distance and/or have a symmetrical orientation about a central location. The capture devices 102 a-102 n configured as a stereo pair may be configured to capture video frames from similar, but slightly different perspectives (e.g., angled inwards to capture fields of view that overlap”). Thus, by using the location of the cameras (a parameter of the camera), the distance in the external environment is able to be found by using images). Thus, it would have been obvious to combine the teachings of Pertsel with Ham. The motivation for doing so would be to use camera parameters to generate depth maps, which can be used to create a 360 degree field of view, which would provide a full view of potential objects that may harm the vehicle. Pertsel teaches in Col. 20, lines 47-58, “In some embodiments, the lenses 112a-112n and/or the capture devices 102 a-102 n may be configured to implement stereo vision. For example, the lenses 112a-112 n and/or the capture devices 102 a-102 n may be arranged to capture multiple perspectives of a location. Using the multiple perspectives, the processors 106a-106n may generate a depth map. The depth map generated by the processors 106a-106n may be used to estimate depth, provide 3D sensing and/or provide an immersive field of view with a 3D effect (e.g., a spherical field of view, an immersive field of view, a 360 degree field of view, less than a 360 degree field of view, etc.)”. Thus, it would have been obvious to combine the teachings of Ham with Pertsel in order to obtain the invention as claimed in Claim 14.
Claims 5 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Ham (US PGPub No 2022/0358800), hereinafter Ham, in view of Pertsel et al. (US Patent No 12304475), hereinafter Pertsel, and further in view of Yasuhiro et al. (JP 2023/076115A), hereinafter Yasuhiro.
As to Claim 5, Ham fails to teach that a neural network that can output a bounding box corresponding to an object, a classification of the object, and an identifier associated with the object.
However, Pertsel teaches a neural network that can output a bounding box corresponding to an object (see Col 15, lines 16-22 ,“The enhanced versions of the signals FRAMES_AFRAMES_N may improve upon the view captured by the lenses 112a-112n ( e.g., provide night vision, provide High Dynamic Range (HDR) imaging, provide more viewing rea, highlight detected objects, provide additional information such as numerical distances to detected objects, provide bounding boxes for detected objects, etc.)” and a classification of the object as a vehicle (see Col. 15, lines 16-22, "The CNN module 150 may determine a likelihood that pixels in the video frames belong to a particular object and/or objects in response to the descriptors. For example, using the descriptors, the CNN module 150 may determine a likelihood that pixels correspond to a particular object (e.g., a person, a vehicle, a car seat, a tree, etc.) and/or characteristics of the object (e.g., a mouth of a person, a hand of a person, headlights of a vehicle, a branch of a tree, a seatbelt of a seat, etc.))”). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Ham with Pertsel. The motivation for doing so would be to identify potential vehicles that may cause harm to a parked vehicle.
Pertsel fails to explicitly teach that an output of a neural network comprises an object identifier associated with the object. However, Yasuhiro teaches that an identifier can be generated for identified objects, (see paragraph [0031], “The specifier 120 outputs information about the specified bounding box BB to the detector 130 as the specification information SI. The class of the object to be monitored may be determined in advance or may be determined by the identification unit 120. The specific information SI includes position coordinates, a size, a class, a likelihood, a tracking ID, and the like of the bounding box BB” and see Fig. 3, where a tracking ID of “BB16-1” is shown for a bounding box identifying a vehicle, and the same ID is shown for the same vehicle in another image). Yasuhiro is combinable with Ham and Pertsel because all three come from the analogous field of image analysis for vehicle detection. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yasuhiro with the teachings of Ham and Pertsel. The motivation for doing so would be to use a bounding box with a specific ID to track the movement of an object. Yasuhiro teaches in paragraph [0047], “In the example shown in FIG. 4, the object enclosed by the bounding box BB21 is a walker. The pedestrian who is standing upright at the time when the image IM-21 is captured is aching on the sidewalk at the time when the image IM-22 is captured. That is, the size of the bounding box BB changes over time.” By using the tracking ID BB21 for the bounding box surrounding a person, the program is able to identify changes in the person by using the tracking ID associated with the bounding box. Thus, it would have been obvious to combine Yasuhiro with the teachings of Ham and Pertsel in order to obtain the invention as claimed in Claim 5.
As to Claim 6, Ham in view of Yasuhiro fails to teach transforming a location in the at least one image that corresponds to the at least the portion of the vehicle into a physical location in the external environment using one or more known parameters of the at least one of the plurality of cameras.
However, Pertsel teaches that camera parameters can be used to calculate distances (see Col. 15, lines 18-39, “The disparity engine 164 may be configured to determine a distance based on images captured as a stereo pair. Two or more of the capture devices 102a-102n may be configured as a stereo pair of cameras. The capture devices 102a-102n configured as a stereo pair may be implemented close to each other at a pre-defined distance and/or have a symmetrical orientation about a central location. The capture devices 102a-102n configured as a stereo pair may be configured to capture video frames from similar, but slightly different perspectives (e.g., angled inwards to capture fields of view that overlap”). Thus, by using the location of the cameras (a parameter of the camera), the distance in the external environment is able to be found by using images). Thus, it would have been obvious to combine the teachings of Pertsel with Ham. The motivation for doing so would be to use camera parameters to generate depth maps, which can be used to create a 360 degree field of view, which would provide full view of potential objects that may harm the vehicle. Pertsel teaches in Col. 20, lines 47-58, “In some embodiments, the lenses 112a-112n and/or the capture devices 102a-102n may be configured to implement stereo vision. For example, the lenses 112a-112n and/or the capture devices 102 a-102 n may be arranged to capture multiple perspectives of a location. Using the multiple perspectives, the processors 106 a-106 n may generate a depth map. The depth map generated by the processors 106 a-106 n may be used to estimate depth, provide 3D sensing and/or provide an immersive field of view with a 3D effect (e.g., a spherical field of view, an immersive field of view, a 360 degree field of view, less than a 360 degree field of view, etc.)”. Thus, it would have been obvious to combine the teachings of Ham with Pertsel in order to obtain the invention as claimed in Claim 6.
Claims 7-10 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Ham (US PGPub No 2022/0358800), hereinafter Ham, in view of Pertsel et al. (US Patent No 12304475), hereinafter Pertsel, and further in view of Yasuhiro et al. (JP 2023/076115A), hereinafter Yasuhiro, and further in view of Yang et al. (US PGPub No 2021/027230), hereinafter Yang.
As to Claim 7, Ham in in view of Pertsel and Yasuhiro fails to explicitly teach that the location in the at least one image that corresponds to the at least the portion of the vehicle comprises a predetermined location on the bounding box.
However, Yang teaches that a location can correspond to a predetermined location on a bounding box (see paragraph [0113], “ A target location may include a location corresponding to a bounding shape, such as a location within a bounding shape, a location along an edge of a bounding shape, or, in some embodiments, a bottom midpoint of a bounding shape (e.g., a point on a bottom, lower edge of a bounding shape). Assuming that the bottom midpoint of the bounding shape is on a ground plane (or a driving surface, with respect to a vehicle or driving environment), the light ray may intersect the ground at this point”). Yang is combinable with Ham, Pertsel, and Yasuhiro, since all three are from the analogous field of image analysis for vehicle detection. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang with the teachings of Ham, Pertsel, and Yasuhiro. The motivation for doing so would be to allow for locations in images to be mapped to location in the external environment. Yang teaches in paragraph [01113], “Given the accuracy of object detection methods (e.g., via a machine learning model(s) 104, via an object detector 708, etc.), tight bounding shapes around objects or obstacles in images may be generated with good accuracy. As such, based on camera calibration parameters, it is possible to compute the path a light ray takes through 3D, world-space, in order to create a pixel in a target location in image-space.” Thus, it would have been obvious to combine the teachings of Yang with the teachings of Ham and Yasuhiro in order to obtain the invention as claimed in Claim 7.
As to Claim 8, Ham fails to teach that a vehicle is determined to be moving based on multiple detections of the object having the object identifier in multiple images from one or more of the plurality of cameras.
However, Pertsel teaches that an object’s movement can be tracked based on multiple detections in different image frames (see Col. 32, lines 66-67, and Col. 33, lines 1-5, “The processors 106a-106n may be configured to predict the trajectory 404a-404 c of the moving object 402 (e.g., estimate the future position of the vehicle 402 based on previously captured video frames) and track the actual movement of the moving object 402 (e.g., detect the actual location of the vehicle 402 based on the currently captured video frames).”
Ham in view of Pertsel and Yang fails to explicitly teach that the object identifier is detected in multiple images. However, Yasuhiro teaches that an identifier can be used to track a bounding box across multiple images (see paragraph [0047], “The pedestrian who is standing upright at the time when the image IM-21 is captured is aching on the sidewalk at the time when the image IM-22 is captured. That is, the size of the bounding box BB changes over time. The determination unit 140 determines, from a change in the size of the bounding box BB, that a special event such as a Uzukumaru [crouching] of a walker in a paroxysmal manner occurs”). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yasuhiro with the teachings of Ham and Pertsel. The motivation for doing so would be to use a bounding box with a specific ID to track the movement of an object, and determine if an anomaly has occurred. Yasuhiro teaches in paragraph [0004],“According to such a conventional technique, it is necessary to learn a normal act and an act to be detected (for example, a shoplifting act) in advance. However, in a case where events to be detected cover a wide range, it is difficult to learn the events to be detected in advance, which is not realistic.” Thus, it would have been obvious to combine Yasuhiro with the teachings of Ham and Pertsel in order to obtain the invention as claimed Claim 8.
As to Claim 9, Ham in view of Pertsel and Yang fails to explicitly teach the neural network is configured to output multiple bounding boxes, including the bounding box, for the multiple detections of the object, and wherein the processing circuitry is configured to determine that the vehicle that is within the threshold distance of the apparatus is moving based on a change in location of the bounding box.
However, Yasuhiro teaches that movement of an object can be determined based on a change in location of the bounding box (see paragraph [0033], “The detection unit 130 detects a change in the bounding box BB in the plurality of images IM. The change of the bounding box BB is a change in position, size, or shape. For example, the detection unit 130 detects a temporal change in at least one of the position, the size, and the shape of the bounding box BB specified at a plurality of time points at which the image IM included in the image information II is captured.”, and paragraph [0036], “The determination unit 140 determines the state of the object based on a change in the bounding box detected by the detection unit 130. The determination unit 140 determines the state of the object on the basis of, for example, movement, staying, deformation, or the like of the object”). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yasuhiro with the teachings of Ham and Pertsel. The motivation for doing so would be to detect events before they occur. Yasuhiro teaches in paragraph [0004], “According to such a conventional technique, it is necessary to learn a normal act and an act to be detected (for example, a shoplifting act) in advance. However, in a case where events to be detected cover a wide range, it is difficult to learn the events to be detected in advance, which is not realistic.” Thus, it would have been obvious to combine Yasuhiro with the teachings of Ham and Pertsel in order to obtain the invention as claimed Claim 9.
As to Claim 10, Ham in view of Pertsel and Yang fails to explicitly teach the neural network is configured to output multiple bounding boxes, including the bounding box, for the multiple detections of the object, and wherein the processing circuitry is configured to determine that the vehicle that is within the threshold distance of the apparatus is moving based on a change in location of the bounding box.
However, Yasuhiro teaches that movement of an object can be determined based on a change in shape of the bounding box (see paragraph [0033], “The detection unit 130 detects a change in the bounding box BB in the plurality of images IM. The change of the bounding box BB is a change in position, size, or shape. For example, the detection unit 130 detects a temporal change in at least one of the position, the size, and the shape of the bounding box BB specified at a plurality of time points at which the image IM included in the image information II is captured”, and paragraph [0036], “The determination unit 140 determines the state of the object based on a change in the bounding box detected by the detection unit 130. The determination unit 140 determines the state of the object on the basis of, for example, movement, staying, deformation, or the like of the object”). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yasuhiro with the teachings of Ham, Pertsel, and Yang. The motivation for doing so would be to detect events before they occur. Yasuhiro teaches in paragraph [0004], “According to such a conventional technique, it is necessary to learn a normal act and an act to be detected (for example, a shoplifting act) in advance. However, in a case where events to be detected cover a wide range, it is difficult to learn the events to be detected in advance, which is not realistic.” Thus, it would have been obvious to combine Yasuhiro with the teachings of Ham, Pertsel and Yang in order to obtain the invention as claimed Claim 10.
As to Claim 15, Claim 15 teaches the same limitations as Claims 5 and 7, and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are analogous to that made in Claims 5 and 7.
As to Claim 16, Claim 16 teaches the same limitation as Claim 6, and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are analogous to that made in Claim 8.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Ham (US PGPub No 2022/0358800), hereinafter Ham, in view of Pertsel et al. (US Patent No 12304475), hereinafter Pertsel, and further in view of Yasuhiro et al. (JP 2023/076115A), hereinafter Yasuhiro, and further in view of Hornung et al. (CN 110497851), hereinafter Hornung.
As to Claim 11, Ham in view of Pertsel and Yasuhiro teaches that a camera may be mounted to a side mirror of a vehicle (see Ham, Fig. 2A, where a camera is located on each mirror). However, Ham fails to teach corresponding a portion of the vehicle into the physical location of the external environment based on a determination of whether the side mirror of the vehicle is in a folded position or an extended position.
However, Hornung teaches that the position of the side mirror can be determined, and a processor can be used to create accurate images even if the side mirror is in a folded or extended position (see paragraph [0009], “A vehicle is provided with which a method for providing prescribed images of the vehicle environment of the vehicle can be implemented, the vehicle typically comprising four cameras. Of the four cameras, one camera is arranged in the front area of the vehicle, one camera is arranged in the rear area, and two cameras are arranged in the two side mirrors. In addition, the vehicle has side mirrors that can be folded. If, for example, the side mirrors are in the unfolded position, the image processing unit of the vehicle can process the individual camera images of the four cameras so that the individual camera images can be combined into an image of the vehicle environment, wherein the vehicle itself is not shown in the image, Instead, the vehicle environment is only shown as desired. If at least one of the two side mirrors currently overlaps, the same image of the vehicle environment is still provided. This is possible because the image processing unit takes into account the deviation of the current camera layout from the reference layout of the cameras, wherein the reference layout in this exemplary embodiment refers to the unfolded position of the side mirrors of the vehicle”). Hornung is combinable with Ham, Pertsel, and Yasuhiro because all three are from the analogous field of image analysis and display in vehicles. Thus, it would have been obvious to one of ordinary skill of the art before the effective filing date of the claimed invention to combine the teachings of Hornung with the teachings of Ham, Pertsel, and Yasuhiro. The motivation for doing so would be to allow for user to fold the side mirrors in tight spaces, but still allow for accurate pictures of the environment around the car to be produced, which is necessary in order to detect possible hit-and-runs. Hornung teaches in in paragraph [0004], “When driving in particular on narrow roads, for example when driving narrowly into a garage, it may be advantageous if the exterior mirrors of the motor vehicle are folded in order to be able to pass through narrow roads without damaging the exterior mirrors. Because each camera of the camera system of the car is usually arranged in the side rearview mirror of the car, the side rearview mirror that utilizes stacking in this driving situation cannot form the same as under the situation that the side rearview mirror unfolds. An image of the vehicle's surroundings, as the arrangement of the cameras relative to the outer walls of the car changes. Currently, when a motor vehicle is driven with folded exterior mirrors, the area of the vehicle environment that is associated with the changed camera position is usually shaded and displayed in black on a corresponding display inside the motor vehicle.” Thus, it would have been obvious to combine the teachings of Hornung with the teachings of Ham, Pertsel and Yasuhiro in order to obtain the invention as taught in Claim 11.
Claims 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ham (US PGPub No 2022/0358800), hereinafter Ham, in view of Pertsel et al. (US Patent No 12304475), hereinafter Pertsel and further in view of Kim et al. (US Pat No 11017620), hereinafter Kim.
As to Claim 17, Ham teaches a method, comprising
using a camera of a vehicle (see paragraph [0053], “The camera 30, which is a module mounted on the vehicle to film a video of a region around the vehicle, may include a front camera 31, a rear camera 32, a first rear-side camera 33, and a second rear-side camera 34”),
that a moving object other than a person is within a threshold distance of the parked vehicle, (see paragraph [0072], “As shown in FIG. 3B, the controller 70 provided in the device configured for recording the drive video of the vehicle according to various exemplary embodiments of the present invention may determine that the dangerous situation has occurred to the host vehicle 200 in a case in which a speed of the rear-side vehicle 300 located within the reference distance (e.g., 5 m) from the host vehicle”, and see paragraph [0073], “That is, the controller 70 may detect the host vehicle 200 and the rear-side vehicle 300 as objects in the first rear-side video or in the second rear-side video, detect a relative speed based on a change in a separation distance for each time between the host vehicle 200 and the rear-side vehicle 300, and estimate the speed of the rear-side vehicle 300 based on the detected relative speed and the collected speed of the host vehicle 200”);
and capturing at least one image using at least the camera of the parked vehicle responsive to the determining, (see paragraph [0009], “device configured for recording a drive video of a vehicle includes a camera configured for filming a front video, a rear video, and a rear-side video of the vehicle, and a controller that detects a dangerous situation of the vehicle based on the rear-side video filmed by the camera, and records the front video, the rear video, and the rear-side video of the vehicle filmed by the camera as an event video in storage in the detected dangerous situation”, and see paragraph [0072], “In the present connection, the controller 70 may store a drive video filmed by the camera 30 in the storage 10”).
Ham fails to teach that the camera is implemented in a parked vehicle to document a potential impact on the parked vehicle by the moving object.
However, Pertsel teaches that moving objects and be detected though a camera (see Col. 1, lines 46-48, “The processor may be configured to perform operations on video frames generated from the pixel data to detect a moving object in the video frame”),
and that the distances of objects can be detected through a camera in a car (see Col. 32, lines 50-56, “Using the knowledge of the alignment and/or orientation (e.g., the distance apart, the angle between the cameras, etc.), the disparity engine 164 may be configured to compare the two images captured and determine the amount of offset between the same object captured by both cameras. The processors 106a-106n may use the amount of offset (e.g., the disparity) to determine a distance to the detected object”),
and teaches that the camera can be implemented in a parked car (see Fig.3, where a parked vehicle is shown, with several cameras) in order to document an accident when the driver is away (see Col 36, lines 29-40, “The processors 106a-106n may be configured to record and/or annotate the video frames. The processors 106a-106n may be configured to select a subset of the video frames (e.g., the video frames that correspond to the event). The annotations 494 may be added to the subset of the video frames. The subset of the video frames with the annotations 494 may be stored in the memory 108. The communication devices 110 may be configured to send the video and/or the annotations to the driver 202 (e.g., a text message alert sent to a smartphone ). For example, the driver 202 may be inside a store shopping and receive the annotated video frame 480 that shows the damage 484 and who caused the damage 484”). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Ham with Pertsel. The motivation for doing so would be to create records of potential accidents when the driver is away from the car. Pertsel teaches in Col. 1, lines 22-39, “As a result, a significant amount of parking accidents occur in parking lots. Often, the accidents happen when one of the cars is unoccupied and parked. The vehicle that gets hit is helpless. The owner of the vehicle that 25 is hit does not even witness the damage until after the collision occurs... It would be desirable to implement automatic collision detection, warning, avoidance and prevention in parked cars.”
Ham in view of Pertsel fails to teach motions sensors in a sleep state.
However, Kim teaches a vehicle with a variety of sensors, including motion sensors, which can be deactivated (see Col. 3 lines 40-46, “The user may also activate or deactivate different sensor(s) of sensor system 10 through a user interface located inside the vehicle configured to communicate to vehicle control system 14 through known wired, wireless, or waveguide manner or an application on a mobile device configured to communicate with the vehicle control system 14 through known wireless or waveguide manner”). Kim also teaches that the optical sensors are always on in order to detect anomalies (see Col. 3, lines 22-24, “The optical sensor of the sensor system 10 will always be active in order to record the vehicle activity during the anomaly. The optical sensor continuously stores optical data onto the memory 16 of the vehicle control system 14. Once one of the sensors of the sensor system 10 senses the anomaly, the vehicle control system 14 will send optical sensor video feed of the affected area of the vehicle to the user display 20”). Kim is combinable with Pertsel because all three are from an analogous field of vehicle safety and image analysis. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kim with Ham and Pertsel. The motivation for doing so would be to reduce unwanted warnings in crowded areas. Kim teaches in Col. 3, Line 46-55, “For example the user may not want to monitor movement around the vehicle when parked in crowded areas since the vehicle control system will be continuously sending unwanted video feed from the optical sensor of sensor system 10, therefore the user may configure the settings on the vehicle user interface or the mobile device such that the vehicle control system 14 will communicate with sensor system 10 to shut off the motion sensors of the sensor system 10 through known wired, wireless, or waveguide manner.” Thus, it would have been obvious to combine the teachings of Kim with the teachings of Ham and Pertsel in order to obtain the invention as claimed in Claim 17.
As to Claim 18, Ham in view of Pertsel and Kim teach teaches that capturing the at least one image comprises capturing an image of an identifier of the moving object (see Pertsel, Col 21, lines 54-64, “For example, the computer vision operations performed by the processors 106a-106n may be configured to detect the vehicles 260a- 260c, determine a location of the vehicles 260a-260c with
respect to the ego vehicle 50 (e.g., distance, direction, relative speed, etc.), determine a make/model of the vehicles 260a-260c, read a license plate of the vehicles 260a-260c,etc. The vehicles 260a-260c may be detected by the computer vision operations if the vehicles 260a-260c are moving (e.g., in traffic) or if the vehicles 260a-260c are stationary( e.g., parked, as shown).” A license plate can serve as an identifier for a moving object. In order for the processors to read the license plate, a picture or video must be taken of the license plate of the moving vehicle). Thus, it would have been obvious to combine the teachings of Pertsel with the teachings of Ham and Kim. The motivation for doing so would have been to allow the driver of the parked vehicle to identify a careless driver, and report the incident. Pertsel teaches in Col. 36, lines 59-61, “In some embodiments, the communication devices 110 may 60 contact the insurance company of the careless driver 482 automatically to confirm the identity and report the incident.” Thus, it would have been obvious to combine the teachings of Pertsel with Ham and Kim in order to obtain the invention as claimed in Claim 18.
As to Claim 19, Ham in view of Pertsel and Kim teach teaches that capturing the at least one image comprises capturing an image of an identifier of the moving object (see Pertsel, Col. 21, lines 54-64, “For example, the computer vision operations performed by the processors 106a-106n may be configured to detect the vehicles 260a- 260c, determine a location of the vehicles 260a-260c with
respect to the ego vehicle 50 (e.g., distance, direction, relative speed, etc.), determine a make/model of the vehicles 260a-260c, read a license plate of the vehicles 260a-260c,etc. The vehicles 260a-260c may be detected by the computer vision operations if the vehicles 260a-260c are moving (e.g., in traffic) or if the vehicles 260a-260c are stationary( e.g., parked, as shown in order for the processors to read the license plate, a picture or video must be taken of the license plate of the moving vehicle, and given to the processors). Thus, it would have been obvious to combine the teachings of Pertsel with the teachings of Ham and Kim. The motivation for doing so would have been to allow the driver of the parked vehicle to identify a careless driver, and report the incident. Pertsel teaches in Col. 36, lines 59-61, “In some embodiments, the communication devices 110 may 60 contact the insurance company of the careless driver 482 automatically to confirm the identity and report the incident.” Thus, it would have been obvious to combine the teachings of Pertsel with Ham and Kim in order to obtain the invention as claimed in Claim 19.
As to Claim 20, Ham fails to teach that determining the moving object is within the threshold distance by performing a geometric determination of a location without identifying individual features of the vehicle.
However, Pertsel teaches an embodiment where distances can be determined using camera parameters, without identifying individual features of the vehicle (see Col. 32, lines 42-56, “One or more of the capture devices 102a-102n may be implemented as a stereo pair of cameras. The processors 106a-106n may utilize the stereo pair of cameras to measure distances to objects such as the vehicles 260a-260c and/or the moving object 402. In an example, each of the capture devices 102a-102n of a stereo pair of cameras may be configured to capture images that are similar but have
slightly different perspectives. Using the knowledge of the alignment and/or orientation (e.g., the distance apart, the angle between the cameras, etc.), the disparity engine 164 may be configured to compare the two images captured and determine the amount of offset between the same object
captured by both cameras. The processors 106a-106n may use the amount of offset (e.g., the disparity) to determine a distance to the detected object”, where using the distance and angles of the cameras in order to calculate distance would be a geometric determination).
Ham in view of Pertsel teaches that proximity sensors may be implemented in the vehicle (see Pertsel, Col. 23, lines 4-8, “The sensor fusion module 152 may be configured to combine the detection of the vehicles 260a-260c from analyzing the video data captured with a detection of objects using the proximity sensors 114a-114d”).
However, Pertsel fails to teach that these proximity sensors may be in a sleep state. However, Kim teaches that sensors used to capture proximity (see Kim, Col. 3, lines 19-21, “The motion sensors will monitor movement of objects or personnel approaching or in close proximity of the vehicle”), can be placed in a sleep state (see Col. 3 lines 40-46, “The user may also activate or deactivate different sensor(s) of sensor system 10 through a user interface located inside the vehicle configured to communicate to vehicle control system 14 through known wired, wireless, or waveguide manner or an application on a mobile device configured to communicate with the vehicle control system 14 through known wireless or waveguide manner”). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kim with Ham and Pertsel. The motivation for doing so would be to reduce unwanted warnings in crowded areas. Kim teaches in Col. 3, lines 46-55, “For example the user may not want to monitor movement around the vehicle when parked in crowded areas since the vehicle control system will be continuously sending unwanted video feed from the optical sensor of sensor system 10, therefore the user may configure the settings on the vehicle user interface or the mobile device such that the vehicle control system 14 will communicate with sensor system 10 to shut off the motion sensors of the sensor system 10 through known wired, wireless, or waveguide manner.” Thus, it would have been obvious to combine the teachings of Kim with the teachings of Ham and Pertsel in order to obtain the invention as claimed in Claim 20.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lynam et al. (US PGPub No 2022/0215671) teaches obtaining an image of an object, and determining if the object is within a threshold distance of a host vehicle.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOUMYA THOMAS whose telephone number is (571)272-8639. The examiner can normally be reached M-F 8:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.T./Examiner, Art Unit 2664
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights