DETAILED ACTION
The amendment filed 01/08/2026, claims 1, 14 and 15 are amended. Claims 6 and 7 are cancelled. Claims 1-5 and 8-15 are pending.
Claims are sufficiently amended to overcome the rejections under 35 U.S.C. 101 Abstract idea and non-statutory computer readable medium claims. Therefore, the rejections are removed.
Response to Arguments
Applicant’s arguments with respect to claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
The Applicant mainly remarks pg. 8-11 – the cited references being not teaching bounding boxes of the object being two sides of each left or right and front or rear, each connected to one side and further the specific shape being trapezoid and rectangle.
The newly cited prior art reference Li et al. (CN 111024040 A) teaches that the object being detected of which bounding box represent each side being left or right side and rear side of the vehicle object and the rear and left or right side are connected by a one line as shown in Fig. 6 of the newly cited prior art reference.
Further, Fig. 6 shows the shape being rectangle elements 682 and 671 being left side and rear sides. And the entirety of rear and left or right side being shape of trapezoid in combination of 3D form of elements 682 and 671. Therefore, the shapes are disclosed in Fig. 6 element 692.
Therefore, the annotation of type of object being detected is also disclosed in fig. as visualization and display output for assistance in navigation.
Therefore, the rejection is maintained. See section below.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5 and 8-15 are rejected under 35 U.S.C. 103 as being unpatentable over Tong et al. (US Pub 20190064841 A1, as provided) in view of Li et al. (Li et al. (CN 111024040 A).
Regarding Claim 1,
Tong discloses A method of annotating a detected object, the method performed by a processor included in a device for annotating the detected object, the method comprising: classifying a class of the object; (Tong, [0007], discloses processor is further configured to extract a feature map of the target object from the context region and determine the confidence level and the bounding box from the extracted feature map. The processor is further configured to determine a plurality of proposal regions for the target object from a plurality of context regions, merge the plurality of proposal regions to form a merged region, and determine the bounding box for the merged region. The processor is further configured to classify the target object in the merged region; image is processed to classify the object within the image)
generating a bounding box comprising for the object based on the class of the object; (Tong, [0036], Figs. 7-9, discloses training a neural network for target vehicle detection. FIG. 7 shows a diagram 700 illustrating a process for selecting image patches for training a neural network. The diagram 700 includes the image 200 of FIG. 2 and several patches extracted from the image 200 for training purposes. Region 702 within image 200 is a region selected to include a target vehicle. The region 702 has a length and height that is substantially equal to the length and height the selected target vehicle. The selected region 702 is referred to as a ground truth positive region. A randomly selected set of patches that include at least a portion of the ground truth positive region 702 are selected, as shown in the image patches of group 704. In addition, a number of background or negative image patches are selected, which do not include any part of the target vehicle. These negative image patches are shown in group 706; object image is divided into patches or regions according to its features and one side and other side of object regions are attached with a line segment) and
generating an annotation on the object in units of the bounding box. (Tong, [0032], [0036], Figs. 7-9, discloses [0032] Once the regions of interest have been pooled, the classifier 310 classifies the object as a target object, i.e., as a target vehicle, pedestrian, cyclist, etc. A bounding box is also predicted for the target object. The bounding box has a width and height substantially equal to those of the target object within the image. Distance to the target object can be determined as well; training a neural network for target vehicle detection. Fig. 7 shows a diagram 700 illustrating a process for selecting image patches for training a neural network. The diagram 700 includes the image 200 of FIG. 2 and several patches extracted from the image 200 for training purposes. Region 702 within image 200 is a region selected to include a target vehicle. The region 702 has a length and height that is substantially equal to the length and height the selected target vehicle. The selected region 702 is referred to as a ground truth positive region. A randomly selected set of patches that include at least a portion of the ground truth positive region 702 are selected, as shown in the image patches of group 704. In addition, a number of background or negative image patches are selected, which do not include any part of the target vehicle. These negative image patches are shown in group 706; object image is processed and classified into specific classes including pedestrian, car etc. and annotated as such as output).
Tong does not explicitly disclose generating a bounding box comprising a first quadrangle and a second quadrangle both sharing one side, for the object based on the class of the object; wherein the first quadrangle is a rectangle, and the second quadrangle is a trapezoid, wherein the first quadrangle corresponds to a front or a rear of the object, and the second quadrangle corresponds to a left side or a right side of the object.
Li discloses generating a bounding box comprising a first quadrangle and a second quadrangle both sharing one side, for the object based on the class of the object; wherein the first quadrangle is a rectangle, and the second quadrangle is a trapezoid, wherein the first quadrangle corresponds to a front or a rear of the object, and the second quadrangle corresponds to a left side or a right side of the object. (Li, Description, Fig. 6, Elements 682, 671, 610, 692, discloses in operation 321, the distance estimation apparatus can detect the bounding box of the target object. For example, the distance estimation apparatus capable of detecting covering the bounding box of the object from the input image. a distance estimation apparatus by using one of various algorithms to detect the boundary frame. For example, the distance estimation device can use neural network to detect with the area comprises a boundary frame corresponding to the object in the input image. neural network can be trained from image output and the detected object (e.g., vehicle) corresponding to the bounding box area. bounding box representation comprises a 2D frame or 3D frame of the object. bounding box can have specific shapes (e.g., rectangular or cuboid) and representation comprises the frame of the space occupied by the object in the 2D space or 3D space; a part of the object can contact each edge of 2D bounding box and 2D bounding box may be defined as the minimum bounding box bounding box to minimize the size of the 2D. the top edge of the 2 D boundary frame can contact the top portion of the object appearing in the input image, and the bottom edge can contact the bottom portion of the object. a part of the object can contact each surface of 3 D boundary frame, and 3D the bounding box may be defined to minimize a size of is the 3D bounding box minimum boundary frame. When the object is a vehicle, the front part of the vehicle capable of contacting the front 3D bounding box, and the rear part of the vehicle can contact the back surface of 3D bounding box. on the top part of the vehicle can contact the 3D bounding box, and the lower part of the vehicle can contact the bottom of the 3D bounding box. side face of side face of the vehicle can contact the 3D bounding box; object information database 340 may include for each object model (e.g., object size information of vehicle model) mapping. object size information can be linked with corresponding information about the size of the object specific model and may include such as the width of the object, a height and a length. The distance estimation apparatus of the embodiment may be based on the visual appearance of objects appearing in the input image determine the object size information corresponding to the object. a distance estimation apparatus capable of obtaining the actual target length from the object size information; rectangular shaped rear end side connected to left side forming trapezoidal shape bounding box and recognized as class car model based on its size and shape determined from database and annotated as that specific model of car)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Tong in view of Li obtaining image of an object and extracting its boundaries and further classifying the object annotation, with the teachings of Li having, by extracting left or right side and rear end side of an object of which are connected by one side and the shape being rectangle and trapezoid of the entire extracted bounding box shape to classify the object in the image being car, person or van in order to explicitly determine shape and size of the object in applications including automatic navigation.
Regarding Claim 2,
The combination of Tong and Irshad further discloses determining attributes of the object, wherein the generating of the annotation comprises generating an annotation regarding the class of the object and the attributes of the object, in units of the bounding box. (Tong, [0032], discloses once the regions of interest have been pooled, the classifier 310 classifies the object as a target object, i.e., as a target vehicle, pedestrian, cyclist, etc. A bounding box is also predicted for the target object. The bounding box has a width and height substantially equal to those of the target object within the image. Distance to the target object can be determined as well; object is classified within the bounding box as vehicle, pedestrian or cyclist etc.). Additionally, the ration and motivation to combine the references Tong and Irshad as applied in rejection of claim 1 apply to this claim.
Regarding Claim 3,
The combination of Tong and Irshad further discloses wherein the class of the object is any one of 'car', 'van', 'truck', 'two-wheeled vehicle', 'pedestrian', 'emergency vehicle', or 'etc.'. (Tong, [0032], discloses once the regions of interest have been pooled, the classifier 310 classifies the object as a target object, i.e., as a target vehicle, pedestrian, cyclist, etc. A bounding box is also predicted for the target object. The bounding box has a width and height substantially equal to those of the target object within the image. Distance to the target object can be determined as well; object is classified within the bounding box as vehicle, pedestrian or cyclist etc.). Additionally, the ration and motivation to combine the references Tong and Irshad as applied in rejection of claim 1 apply to this claim.
Regarding Claim 4,
The combination of Tong and Irshad further discloses wherein, in a case where the object is a car designed to transport goods and is loaded with another vehicle, a class of the other vehicle is not classified. (Tong, [0006-0008], discloses a system for driving an autonomous vehicle is disclosed. The system includes a camera configured to obtain an image of a surrounding region of the vehicle, an actuation device for controlling a parameter of motion of the vehicle, and a processor. The processor is configured to select a context region within the image, the context region including a detection region therein, estimate a confidence level indicative of the presence of at least a portion of the target object in the detection region and a bounding box associated with the target object, determine a proposal region from the bounding box when the confidence level is greater than a selected threshold, determine a parameter of the target object within the proposal region, and control the actuation device to alter a parameter of motion of the vehicle based on the parameter of the target object; processor is further configured to extract a feature map of the target object from the context region and determine the confidence level and the bounding box from the extracted feature map. The processor is further configured to determine a plurality of proposal regions for the target object from a plurality of context regions, merge the plurality of proposal regions to form a merged region, and determine the bounding box for the merged region. The processor is further configured to classify the target object in the merged region; the camera includes a camera at a front end of the vehicle, a camera at a rear end of the vehicle, or a camera at a front end of the vehicle and a camera at a rear end of the vehicle. The processor is further configured to track movement of a plurality of temporally spaced bounding boxes to determine a movement of the target object. The processor is further configured to determine a velocity of the target object across a line of sight of the vehicle). Additionally, the ration and motivation to combine the references Tong and Irshad as applied in rejection of claim 1 apply to this claim.
Regarding Claim 5,
The combination of Tong and Irshad further discloses wherein the generating of the bounding box comprises generating the bounding box by applying different criteria depending on the classified class of the object. (Tong, [0032], discloses once the regions of interest have been pooled, the classifier 310 classifies the object as a target object, i.e., as a target vehicle, pedestrian, cyclist, etc. A bounding box is also predicted for the target object. The bounding box has a width and height substantially equal to those of the target object within the image. Distance to the target object can be determined as well; bounding boxes are generated according to different classes of objects). Additionally, the ration and motivation to combine the references Tong and Irshad as applied in rejection of claim 1 apply to this claim.
Regarding Claim 8,
The combination of Tong and Irshad further discloses wherein, in a case where an upper surface of the object is exposed, the first quadrangle or the second quadrangle comprises the upper surface of the object. (Tong, [0005], [0032], discloses once the regions of interest have been pooled, the classifier 310 classifies the object as a target object, i.e., as a target vehicle, pedestrian, cyclist, etc. A bounding box is also predicted for the target object. The bounding box has a width and height substantially equal to those of the target object within the image. Distance to the target object can be determined as well; the camera includes a camera at a front end of the vehicle, a camera at a rear end of the vehicle, or a camera at a front end of the vehicle and a camera at a rear end of the vehicle. Movement of a plurality of temporally spaced bounding boxes is tracked in order to determine a movement of the target object. A velocity of the target object across a line of sight of the vehicle is determined; bounding boxes are generated according to different classes of objects and determining shape of bounding box is design preference of any shape). Additionally, the ration and motivation to combine the references Tong and Irshad as applied in rejection of claim 1 apply to this claim.
Regarding Claim 9,
The combination of Tong and Irshad further discloses wherein, in a case where the object is a wheeled vehicle, the second quadrangle comprises a line segment connecting wheel-ground points to each other. (Tong, [0032], discloses once the regions of interest have been pooled, the classifier 310 classifies the object as a target object, i.e., as a target vehicle, pedestrian, cyclist, etc. A bounding box is also predicted for the target object. The bounding box has a width and height substantially equal to those of the target object within the image. Distance to the target object can be determined as well; bounding boxes are generated according to different classes of objects and determining shape of bounding box is design preference of any shape). Additionally, the ration and motivation to combine the references Tong and Irshad as applied in rejection of claim 1 apply to this claim.
Regarding Claim 10,
The combination of Tong and Irshad further discloses wherein, in a case where the class of the object is 'two-wheeled vehicle' or 'pedestrian', a width of the first quadrangle is generated to be equal to a width of a shoulder of a person included in the object. (Tong, [0032], discloses once the regions of interest have been pooled, the classifier 310 classifies the object as a target object, i.e., as a target vehicle, pedestrian, cyclist, etc. A bounding box is also predicted for the target object. The bounding box has a width and height substantially equal to those of the target object within the image. Distance to the target object can be determined as well; bounding boxes are generated according to different classes of objects and determining shape of bounding box is design preference of any shape). Additionally, the ration and motivation to combine the references Tong and Irshad as applied in rejection of claim 1 apply to this claim.
Regarding Claim 11,
The combination of Tong and Irshad further discloses in a case where the detected object is a vehicle and a proportion of the object visible in image data is less than a threshold proportion of a total size of the object, determining not to generate the bounding box. (Tong, [0032], [0006-0008], discloses a system for driving an autonomous vehicle is disclosed. The system includes a camera configured to obtain an image of a surrounding region of the vehicle, an actuation device for controlling a parameter of motion of the vehicle, and a processor. The processor is configured to select a context region within the image, the context region including a detection region therein, estimate a confidence level indicative of the presence of at least a portion of the target object in the detection region and a bounding box associated with the target object, determine a proposal region from the bounding box when the confidence level is greater than a selected threshold, determine a parameter of the target object within the proposal region, and control the actuation device to alter a parameter of motion of the vehicle based on the parameter of the target object; processor is further configured to extract a feature map of the target object from the context region and determine the confidence level and the bounding box from the extracted feature map. The processor is further configured to determine a plurality of proposal regions for the target object from a plurality of context regions, merge the plurality of proposal regions to form a merged region, and determine the bounding box for the merged region. The processor is further configured to classify the target object in the merged region; the camera includes a camera at a front end of the vehicle, a camera at a rear end of the vehicle, or a camera at a front end of the vehicle and a camera at a rear end of the vehicle. The processor is further configured to track movement of a plurality of temporally spaced bounding boxes to determine a movement of the target object. The processor is further configured to determine a velocity of the target object across a line of sight; once the regions of interest have been pooled, the classifier 310 classifies the object as a target object, i.e., as a target vehicle, pedestrian, cyclist, etc. A bounding box is also predicted for the target object. The bounding box has a width and height substantially equal to those of the target object within the image. Distance to the target object can be determined as well; bounding boxes are generated according to different classes of objects and determining shape of bounding box is design preference of any shape). Additionally, the ration and motivation to combine the references Tong and Irshad as applied in rejection of claim 1 apply to this claim.
Regarding Claim 12,
The combination of Tong and Irshad further discloses wherein the attributes of the object comprises visibility, a movement state, a lane position, a major state, a size and a subclass of the object. (Tong, [0027], [0032], discloses vehicle 102 further includes a processor 116 that performs methods of vehicle navigation and of detection of target objects. The processor 116 receives one or more images from the cameras 104a, 104b and 104c, locates and classifies a target object within the one or more images and determines parameters of motion of the target object. The parameters may include a location, angular location, velocity of the target object, for example. The processor 116 can predict an outcome of driving the vehicle 102 based on the parameters of the target object and the internal state parameters of the vehicle 102 and can calculate and implement an updated internal state for providing a different outcome. For example, the processor 116 can determine that based on the location and velocity of the target object and based on the direction and velocity of vehicle 102 an impact with the target object is imminent. The processor 116 can then send a control signal to the actuation devices 112 in order to change a parameter of motion such as a speed or direction of the vehicle 102 in order to avoid impact with the target object. The processor 116 further performs methods for detecting and tracking target objects as disclosed herein; once the regions of interest have been pooled, the classifier 310 classifies the object as a target object, i.e., as a target vehicle, pedestrian, cyclist, etc. A bounding box is also predicted for the target object. The bounding box has a width and height substantially equal to those of the target object within the image. Distance to the target object can be determined as well; bounding boxes are generated according to different classes of objects and determining shape of bounding box is design preference of any shape). Additionally, the ration and motivation to combine the references Tong and Irshad as applied in rejection of claim 1 apply to this claim.
Regarding Claim 13,
The combination of Tong and Irshad further discloses controlling an ego vehicle based on the generated annotation. (Tong, [0032], discloses once the regions of interest have been pooled, the classifier 310 classifies the object as a target object, i.e., as a target vehicle, pedestrian, cyclist, etc. A bounding box is also predicted for the target object. The bounding box has a width and height substantially equal to those of the target object within the image. Distance to the target object can be determined as well; bounding boxes are generated according to different classes of objects and determining shape of bounding box is design preference of any shape). Additionally, the ration and motivation to combine the references Tong and Irshad as applied in rejection of claim 1 apply to this claim.
Claims 14 and 15 recite device with elements corresponding to the method steps recited in Claim 1. Therefore, the recited elements of the device Claim 14 are mapped to the proposed combination in the same manner as the corresponding steps of Claim 1. Additionally, the rationale and motivation to combine the Tong and Irshad references presented in rejection of Claim 1, apply to these claims.
Furthermore, the combination of Tong and Irshad further discloses A device for annotating a detected object, the device comprising: a memory storing at least one program; and a processor configured to execute the at least one program and A computer-readable recording medium having recorded thereon a program. (Tong, [0006-0008], discloses a system for driving an autonomous vehicle is disclosed. The system includes a camera configured to obtain an image of a surrounding region of the vehicle, an actuation device for controlling a parameter of motion of the vehicle, and a processor. The processor is configured to select a context region within the image, the context region including a detection region therein, estimate a confidence level indicative of the presence of at least a portion of the target object in the detection region and a bounding box associated with the target object, determine a proposal region from the bounding box when the confidence level is greater than a selected threshold, determine a parameter of the target object within the proposal region, and control the actuation device to alter a parameter of motion of the vehicle based on the parameter of the target object; processor is further configured to extract a feature map of the target object from the context region and determine the confidence level and the bounding box from the extracted feature map. The processor is further configured to determine a plurality of proposal regions for the target object from a plurality of context regions, merge the plurality of proposal regions to form a merged region, and determine the bounding box for the merged region. The processor is further configured to classify the target object in the merged region; the camera includes a camera at a front end of the vehicle, a camera at a rear end of the vehicle, or a camera at a front end of the vehicle and a camera at a rear end of the vehicle. The processor is further configured to track movement of a plurality of temporally spaced bounding boxes to determine a movement of the target object. The processor is further configured to determine a velocity of the target object across a line of sight of the vehicle).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Sriram et al. (US Pub No. 20190294889 A1, approaches for smart area monitoring suitable for parking garages or other areas. These approaches may include ROI-based occupancy detection to determine whether particular parking spots are occupied by leveraging image data from image sensors, such as cameras. These approaches may also include multi-sensor object tracking using multiple sensors that are distributed across an area that leverage both image data and spatial information regarding the area, to provide precise object tracking across the sensors. Further approaches relate to various architectures and configurations for smart area monitoring systems, as well as visualization and processing techniques. For example, as opposed to presenting video of an area captured by cameras, 3D renderings may be generated and played from metadata extracted from sensors around the area, Abstract, Fig. 1C)
Turn et al. (US 20130321627 A1, [0070], order to prevent a moving vehicle from departing a surface, road or path, information from the entire field of view in front of the vehicle is not required. The drivable surface, road or highway, ahead of the vehicle is important. An important truism for a RDSS application is that the road edges tend to meet at a vanishing point beyond the horizon. The important region of the image is the portion that is immediately ahead of the vehicle to the horizon, as depicted in FIG. 10. This region, bounded by the road edges, is generally in the shape of a trapezoid. This trapezoid shape can be approximated by regions of interest (ROI) superimposed on a two-dimensional image, depicted in FIG. 11, that are labeled ROI C1, ROI C2, and ROI C3. Areas outside the depicted ROI can be ignored in order to reduce the processing demands, or only considered when evaluating the image as a whole for exposure evaluation to determine if an image is acceptable for further edge detection analysis)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PINALBEN V PATEL whose telephone number is (571)270-5872. The examiner can normally be reached M-F: 10am - 8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at 571-272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Pinalben Patel/Examiner, Art Unit 2673