DETAILED ACTION
Information Disclosure Statement
The information disclosure statements submitted on 02/25/2025 and 03/04/2025 have been considered by the Examiner and made of record in the application file.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-7, 9-10, 12-14, 16-17 and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Evans (US 2024/0242514 A1) in view of Tateno (US 2023/0168686 A1).
Regarding claims 1, 10 and 17, Evans discloses a method of monitoring cargo that is moving within a work area (Evans discloses an autonomous vision system that monitors cargo as the cargo is being loaded onto a vehicle, including an aircraft, and as the cargo moves through the aircraft opening and into the interior space, see paragraphs 45 and 48-50.), the method comprising:
[claim 10: A computing device configured to monitor cargo within a work area, the computing device comprising: processing circuitry; and memory circuitry comprising program instructions therein that, when executed by the processing circuity, configures the computing device to: (paragraph 96)]
[claim 17: A computing device configured to monitor cargo within a work area, the computing device comprising: processing circuitry; and memory circuitry comprising program instructions therein that, when executed by the processing circuity, configures the computing device to: (paragraph 96)]
capturing images of the cargo with cameras as the cargo moves through the work area; (Evans discloses cameras 70 positioned at various locations around the vehicle opening that capture images and video of cargo 200 as the cargo is moved through opening 104. Evans further discloses that the cameras have fixed fields of view and capture a sequence of images showing the cargo moving across the field of view, see paragraphs 50-52.)
detecting the cargo in the images; (Evans discloses that computing device 20 performs cargo detection on images received from cameras 70. The cargo is detected using semantic segmentation and a bounding box, where pixels corresponding to the cargo are identified separately from pixels that are not part of the cargo, see paragraphs 57-61.)
generating an approximation of the cargo in the images (Evans discloses forming bounding box 73 that encapsulates the identified cargo, see paragraph 61. However, Evans does not disclose that the bounding box itself is a 3D bounding box represented in a 3D coordinate system
tracking a position of the cargo and movement of the cargo through the work area based on a location (Evans discloses identifying a point on the cargo and tracking the cargo through successive images as the cargo moves through the field of view of cameras 70. Evans further discloses using timestamps to track movement of the cargo through the different images, see paragraphs 62-64. However, Evans does not disclose tracking the cargo based specifically on a location of a 3D bounding box within a 3D Cartesian frame.)
generating a three-dimensional mesh model of the cargo. (Evans discloses determining a 3D volumetric mesh of the cargo from the captured images. Evans estimates a sparse 3D point cloud from matched visual features and matches a 3D mesh to the points, following by mesh closure to complete the 3D model, see paragraphs 76-78.)
Evans fails to specifically disclose generating an approximation of the cargo in the images with a 3D bounding box and using a location of that 3D bounding box within a 3D Cartesian frame to determine cargo position.
In related art, Tateno discloses generating an approximation of the cargo in the images with a 3D bounding box that extends around the cargo (Tateno discloses estimating the contour shape of cargo 15 as a 3D bounding box having three positional parameters and three size parameters. Tateno determines an axis-aligned bounding box AABB having height, width, depth, and a barycentric position of the cargo, see paragraph 45. Tateno further discloses generating a 3D point group from the captured depth image, converting the point group from a cargo imaging coordinate system into a vehicle coordinate system, and determining minimum and maximum coordinate values along each of the three axes to obtain the AABB surrounding the cargo, see paragraph 46.) and tracking a position of the cargo and movement of the cargo through the work area based on a location within a 3D Cartesian frame of the bounding box in the images (Tateno disclose a 3D coordinate system having X, Y and Z axes and converts 3D cargo information between an imaging coordinate system, a vehicle coordinate system and a world coordinate system. The AABB includes the barycentric position of the cargo in the 3D coordinate system, see paragraphs 38-39 and 45-46. Tateno further teaches a monitoring camera having a known position and orientation in the world coordinate system and converting observed cargo shape and position into the vehicle coordinate system, see paragraph 54.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Tateno into the teachings of Evan to effectively permit its position to be consistently represented and tracked as the cargo moves between the fields of the cameras.
Regarding claim 3, Evans, as modified by Tateno, discloses the claimed invention wherein identifying the cargo from the images and classifying the cargo. (Evans: paragraphs 57-58, Tateno: paragraph 60)
Regarding claim 4, Evans, as modified by Tateno, discloses the claimed invention wherein positioning the bounding box around a perimeter of the cargo in the images. (Evans: paragraph 61; Tateno paragraphs 45-60)
Regarding claim 5, Evans, as modified by Tateno, discloses the claimed invention wherein determining coordinates for corners of the bounding box in the images. (Tateno paragraph 46)
Regarding claim 6, Evans, as modified by Tateno, discloses the claimed invention wherein determining local coordinates of the bounding box from the images; converting the local coordinates into the three-dimensional Cartesian frame; (Tateno, paragraphs 34 and 45-46) and tracking a position of the cargo in the work area based on the Cartesian frame. (Evans, paragraphs 62-64)
Regarding claim 7, Evans, as modified by Tateno, discloses the claimed invention wherein capturing the images of the cargo comprises capturing the images while the cargo is being loaded onto an aircraft. (Evans, paragraphs 48-50)
Regarding claim 9, Evans, as modified by Tateno, discloses the claimed invention wherein determining positions of the bounding box relative to a local coordinate system of the cameras in the images; and converting the positions in the local coordinate system to the three- dimensional Cartesian frame and tracking the cargo relative to the three dimensional Cartesian frame. (Evans, paragraphs 62-64; Tateno: paragraphs 34 and 46)
Regarding claim 12, Evans, as modified by Tateno, discloses the claimed invention wherein cameras that are configured to be mounted in different points in the work area, the cameras configured to capture the images of the cargo from different perspectives. (Evans paragraph 51; Tateno paragraph 53)
Regarding claim 13, Evans, as modified by Tateno, discloses the claimed invention wherein the cameras are configured to be mounted at a door of a vehicle. (Evans paragraphs 50-51)
Regarding claim 14, Evans, as modified by Tateno, discloses the claimed invention wherein the processing circuitry is further configured to classify the cargo as being from one of a plurality of different classes based on the images. (Tateno paragraph 60)
Regarding claim 16, Evans, as modified by Tateno, discloses the claimed invention wherein communication circuitry configured to upload through a communication network a position of the cargo in the work area. (Evans paragraphs 12 and 86-90)
Regarding claim 19, Evans, as modified by Tateno, discloses the claimed invention wherein the processing circuitry is further configured to: determine a position on the bounding box; and track the location of the cargo in the work area based on the position on the bounding box. (Evans paragraph 62; Tateno paragraphs 45-46)
Regarding claim 20, Evans, as modified by Tateno, discloses the claimed invention wherein the computing device is integrated within an aircraft and the processing circuitry is further configured to operate an aspect of the aircraft during flight. (Evans paragraphs 55 and 90)
Claims 2 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Evans in view of Tateno and in further view of Tovchigrechko (US 2024/0153223 A1).
Regarding claims 2 and 15, Evans, as modified by Tateno, discloses the claimed invention except for wherein classifying the cargo into one of a plurality of classifications; and generating the three-dimensional mesh model of the cargo for just one of the plurality of classifications.
In related art, Tovchigrechko discloses classifying the cargo into one of a plurality of classifications; and generating the three-dimensional mesh model of the cargo for just one of the plurality of classifications. (paragraph 55)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Tovchigrechko into the teachings of Evans and Tateno to effectively exclude unrelated objects and improve object specific 3D reconstruction.
Claims 8 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Evans in view of Tateno and in further view of Kankaanpaa (US 2019/0230307 A1).
Regarding claim 8, Evans, as modified by Tateno, discloses the claimed invention except for wherein determining that the cargo is not included in the images that were captured after a time period; and determining that the cargo has moved out of the work area after the time period.
In related art, Kankaanpaa discloses determining that the cargo is not included in the images that were captured after a time period; and determining that the cargo has moved out of the work area after the time period. (paragraph 26)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Kankaanpaa into the teachings of Evans and Tateno to effectively distinguish cargo leaving the monitored area from a temporary missed detection.
Regarding claim 11, Evans, as modified by Tateno, discloses the claimed invention except for wherein the processing circuitry is configured to determine that the cargo has moved out of the work area after determining that the cargo does not appear in a predetermined number of the images.
In related art, Kankaanpaa discloses determining determine that the cargo has moved out of the work area after determining that the cargo does not appear in a predetermined number of the images. (paragraph 26)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Kankaanpaa into the teachings of Evans and Tateno to effectively distinguish cargo leaving the monitored area from a temporary missed detection.
Claims 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Evans in view of Tateno and in further view of Petrov (US 2002/0050988 A1) and Frakes (US 2005/0041842 A1).
Regarding claim 18, Evans, as modified by Tateno, discloses the claimed invention except for trace contours of the cargo in the images; create masked images that mask the part of the image corresponding to the cargo based on the contours; and reconstruct the cargo based on the masked images.
In related art, Petrov discloses trace contours of the cargo in the images. (paragraphs 142-143)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Petrov into the teachings of Evans and Tateno to more accurately define the boundary of the detected cargo prior to 3D reconstruction.
Furthermore, in related art, Frakes discloses creating masked images that mask the part of the image corresponding to the cargo based on the contours; (paragraphs 30-31 and 41) and reconstruct the cargo based on the masked images. (paragraph 46)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Frakes into the teachings of Evans, Tateno and Petrov to isolate the cargo from background image data and effectively improve the accuracy of the resulting 3D reconstruction.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BOBBAK SAFAIPOUR whose telephone number is (571)270-1092. The examiner can normally be reached Monday - Friday, 8:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BOBBAK SAFAIPOUR/Primary Examiner, Art Unit 2665
Prosecution Insights