Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant’s submission filed 6/25/26 has been entered. Claims 1-20 are presented for examination.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 9/26/25 and 7/17/25 are being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6-11, 13-18, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Grundfest et al. (US 20200254619 A1), in view of ROSAS-MAXEMIN et al. (WO 2023107584), in further view of Strickland et al. (US 5984503 A).
Re-claim 1, Grundfest et al. teach --a system for visualizing inventory of an intermodal container yard, the system comprising:
an inventory database storing a unit inventory, the unit inventory comprising information about a plurality of shipping containers physically located in the intermodal container yard;
(see e.g. [0048] - database of containership 106).
[0049] In an embodiment, the information system 113 may query a database of a containership 106 via a designated interface for the manifest of a previously identified container. Alternatively, the container may also be identified by its position on the containership 106 deck.)
one or more memory units; and
one or more computer processors communicatively coupled to the inventory database and the one or more memory units and configured to:
access the unit inventory stored in the inventory database;
(see e.g. [0049] In an embodiment, the information system 113 may query a database of a containership 106 via a designated interface for the manifest of a previously identified container. Alternatively, the container may also be identified by its position on the containership 106 deck.)
-provide one or more instructions to display, on an electronic display of a user computing system, a graphical user interface comprising a three-dimensional representation of the intermodal container yard;
(see e.g. [0101] Safety management of a smart container yard 100 may be a challenging task. Multiple manned, autonomous and teleoperated vehicles, as well aerial drones, containers in transfer and crane arms create a complex three-dimensional interplay that must be managed in a manner as to prevent collisions.
[0102] The container yard information system 113 may generate a space state representing physical objects in the container yard 100 and generate an operational plan based on the space state to include navigation paths that avoid physical objects. -=- . For example, responsive to the data acquired from the sensor arrays 107, the information system 113 subsequently builds and maintains a representation of the container yard 100 space as a cuboid populated by polygonal shapes enveloping physical objects in the container yard 100, further referred to as obstacles Ω. The information system 113 subsequently performs a 3D constrained Delaunay triangulation of vertices of obstacles ν.sub.i in the yard, which also includes edges of obstacles ϵ.sub.i as a part of the triangulation based on this representation of the space of the container yard. 3D Delaunay triangulations possess the empty sphere property, i.e. the circumscribing sphere of each cell produced by triangulation process contains no other vertices of the triangulation in its interior.
-update, in real time, the three-dimensional representation of the intermodal container yard based on the information about the plurality of shipping containers in the unit inventory,
(see e.g. [0102] the information system 113 subsequently builds and maintains a representation of the container yard 100 space as a cuboid populated by polygonal shapes enveloping physical objects in the container yard 100, further referred to as obstacles Ω. The information system 113 subsequently performs a 3D constrained Delaunay triangulation of vertices of obstacles ν.sub.i in the yard, which also includes edges of obstacles ϵ.sub.i as a part of the triangulation based on this representation of the space of the container yard. 3D Delaunay triangulations possess the empty sphere property, i.e. the circumscribing sphere of each cell produced by triangulation process contains no other vertices of the triangulation in its interior.
[0104] In an embodiment, the onboard computing system of a utility vehicle such as a forklift 114 or a container handler may sequentially transmit updates on forklift or spreader extension level to the container yard information system 113. This information may then be used to update the dynamic 3D obstacle map and path planning of other vehicles and drones.)
wherein updating the three-dimensional representation of the intermodal container yard comprises:
providing, in the three-dimensional representation of the intermodal container yard, a visual representation of each shipping container physically located in the intermodal container yard,
(see e.g. [0025] A smart container yard information system also maintains a representation of the current state of a container yard with the degree of precision and completeness desirable for safe operation utilizing a distributed array of stereo optical and infrared cameras, ranging devices such as sonars, LIDARs, radars, BLE beacons or other implements as well as computational systems for processing sensor measurements. A container yard space state may be represented as a cuboid populated by convex polygonal shapes enveloping physical objects in the container yard. Vehicles, structures and other objects may be identified by means such as Bluetooth low energy (BLE) beacons, radio frequency identification (RFID) tags, near field communication (NFC) tags, QR codes or other machine-readable decals, computer vision algorithms applied to human-readable IDs such as license plates or vehicle numbers.)
--providing, in the three-dimensional representation of the intermodal container yard, a visual representation of a current location of a vehicle travelling within the intermodal container yard; and
(see e.g. [0106] In an embodiment, the container yard information system 113 acquires information necessary to update dynamic obstacle map on positions and shapes of obstacles such as forklifts 114, yard trucks 115, aerial drones 116, wireless transmitters 109 and cranes 102, 103 from sensor arrays 107 such as optical or infrared camera arrays and computer vision techniques, or ranging sensors such as LIDARs, sonars or radars.)
[0028] In another embodiment, path homotopies in a dynamical 3D environment of a container yard are identified using methods such as constrained Delaunay triangulation, and coordinated path planning for transfer cranes, ground and aerial vehicles in the container yard is executed. Path homotopies may be refined to propose safe path solutions, which may subsequently be provided to human operators using methods such as augmented reality displays, or to machine intelligence agents via appropriate channels and protocols.)
Although Grundfest et al. anticipate wherein the visual representation of each particular shipping container is placed at a virtual parking location in the three-dimensional representation of the intermodal container yard that corresponds to a physical parking location of the particular shipping container in the intermodal container yard;
(see e.g. claim 12 - wherein the augmented reality view includes a representation of information from the container yard information system describing physical objects in a vicinity of the truck.)
[0064] A number of methods of preparing information for teleoperator perception may be employed. Some such methods may involve overlays of text, geometric shapes, highlights and other augmented reality elements onto video feeds received from cameras in the container yard 100 to provide an augmented reality view of a perspective of the truck that may generate HUDs (heads-up displays) on a display device at the teleoperator workstation 117. -- Video feeds comprising a main teleoperator view may occupy the whole background of a monitor, while additional data such as virtual lane boundaries to follow, lines enveloping the expected vehicle position for a brief future time interval, representations of objects in a vicinity of the vehicle, shipment manifests, additional exocentric camera feeds or the speed limit to observe may be rendered on top of the main teleoperator view. In another embodiment, the HUD of the teleoperator workstation 117 is implemented as a projection of auxiliary information on the screen of a virtual reality headset.)
ROSAS-MAXEMIN et al. explicitly teach --wherein the visual representation of each particular shipping container is placed at a virtual parking location in the three-dimensional representation of the intermodal container yard that corresponds to a physical parking location of the particular shipping container in the intermodal container yard; and
(see e.g. [0019] As discussed above, in some examples computer 220 may use machine learning and computer vision algorithms on image and video data captured by camera 210 to identify shipping containers and open parking spots in a container yard. The next remaining step may be determining the location of the shipping containers and open parking spots in a container yard.
[0015] The container yard 100 includes one or more shipping containers 160, one or more parking spots 130, remote location 140, yard rig 120, and drone 110. The yard rig 120 and drone 150 each include locator system 110. In some aspects, the size (e.g., dimensions in three-dimensional space) of container yard 100 may vary depending on the number of shipping containers 160 and open parking spots 130. )
ROSAS-MAXEMIN et al. also teach -- a system for visualizing inventory of an intermodal container yard,
(see abstract --- Described are systems and techniques for locating and identifying shipping containers or open parking spots in a container yard using computer vision and machine learning algorithms, a camera, and position, navigation, and timing (PNT) sensors.
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Grundfest et al., and include the steps cited above, as taught by ROSAS-MAXEMIN et al., in order to locate and identify all the shipping containers and open parking spots (see e.g. [0020]).
Although anticipated by both Grundfest et al., (see e.g. [0084] [0043]) and ROSAS MAXEMIN et al. (see e.g. [0015], [0016] ), Strickland et al. explicitly teach
--wherein the graphical user interface is configured to permit a user to select visual representations of shipping containers and to select virtual parking locations of shipping containers within the three-dimensional representation of the intermodal container yard;
provide one or more instructions to display in the graphical user interface, in response to a user selection in the three-dimensional representation of the intermodal container yard of a particular visual representation of a particular shipping container or a particular virtual parking location, metadata about the particular shipping container.
(see e.g. (col. 10, lines 23-34 --The pallet list comprises a series of buttons 324, each button having pallet information for a single pallet displayed thereon. If the user wishes to view additional pallet information about a pallet, such as the information regarding setups located on a pallet, the user may select with a selection device, either a pallet button 324, or pallet icon 322, corresponding to the pallet in which the user is interested. Upon selection of a pallet, it is preferred that cross hairs 326 are displayed on the pallet selected. Once a pallet is selected, additional pallet information can be displayed on the display device, such as offset coordinates 325 for the pallet.
(col. 9, lines 64-67; col. 10, lines 1-7 -Then, when the user enters a pallet location, such as by using dialog box 306 of FIG. 3, the program matches the pallet location entered with a pallet park station number, and displays a pallet icon, such as pallet icon 322, at the icon location selected by the user for that park station number during the initial configuration. Thus, machine configuration image 308 indicates the actual configuration of the particular machining system utilized by the user, and pallet icons 322 are overlayed on the image in order to indicate at which pallet park stations pallets are actually located.
(col. 9, lines 16-20 --Preferably, the machine configuration image 308 is drawn using a graphics program or CAD program, and is stored in a graphics format directly supported by WINDOWS, such as in a .PCX, .DXF, or .BMP format. )
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Grundfest et al., in view of ROSAS-MAXEMIN et al., and include the steps cited above, as taught by Strickland et al., in order to quickly and efficiently view the status of a plurality of pallets (see e.g. col. 4, lines 26-27).
Re-claim 2, Grundfest et al. teach --The system for visualizing inventory of the intermodal container yard of Claim 1, wherein the metadata about the particular shipping container comprises one or more of:
an indication of a particular physical parking location of the particular shipping container;
a current image of the particular physical parking location;
a current carrier identification of the particular shipping container;
a current identification number of the particular shipping container;
a timestamp of the current image.
(see e.g. [0102] In an embodiment the container yard information system 113 may utilize a distributed array 107 of sensors such as stereo cameras, ranging devices such as sonars, LIDARs, or radars, BLE beacons, NFC tags and other implements to maintain a representation of the current state of the container yard 100 with the required degree of precision and completeness. The container yard information system 113 may generate a space state representing physical objects in the container yard 100 and generate an operational plan based on the space state to include navigation paths that avoid physical objects.)
[0045] In an embodiment, the information system 113 may apply computer vision methods to read a license plate of the carrier truck 103 carrying a container and subsequently query a database of vehicles allowed entry to the container yard 100 for the license plate number parsed in order to perform container identification.
[0105] In an embodiment, the onboard computing system of a utility vehicle such as a forklift 114 or a container handler may transmit the commencement timestamp of extension or other operations changing the shape of a utility vehicle and metadata describing the process such as extension speed and the expected extension length to the container yard information system 113.)
Re-claim 3, Grundfest et al. teach --The system for visualizing inventory of the intermodal container yard of Claim 2, wherein the metadata about the particular shipping container further comprises one or more of:
a previous image of the particular physical parking location;
a previous carrier identification of the particular shipping container;
a previous identification number of the particular shipping container; and
a timestamp of the previous image.
(see e.g. 0049] In an embodiment, the information system 113 may query a database of a containership 106 via a designated interface for the manifest of a previously identified container. Alternatively, the container may also be identified by its position on the containership 106 deck.)
Re-claim 4, Grundfest et al. do not teach the limitation as claimed.
However, ROSAS-MAXEMIN et al. teach --The system for visualizing inventory of the intermodal container yard of Claim 2, wherein the indication of the particular physical parking location of the particular shipping container comprises:
a lot identification;
a row identification; and
a spot identification.
(see e. g. [0017] In some examples, the designated set of coordinates or location (e.g., latitude and longitude) of every shipping container and parking spot (e.g., one or more shipping containers 160 and one or more parking spots 130 as illustrated in FIG. 1) may be stored (e.g., stored in memory of computer 220) in locator system 200 and accessible by computer 220.
[0018] Examples of open parking spot identification features may include, but are not limited to, alphanumeric identifiers in the parking spot and parking lot striping features. -- In some examples, the computer 220 may use image and video data captured by camera 210 and process the data using computer vision and machine learning algorithms, which may be trained to detect particular features of the surrounding environment such as shipping container and open parking spot identification features.
[0020] As discussed above, in some examples, the location may be a geographic coordinate using latitude and longitude or another type of coordinate system to indicate the location of the shipping containers in the yard. The same principle may apply to determine the location of open parking spots.)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Grundfest et al., and include the steps cited above, as taught by ROSAS-MAXEMIN et al., in order to locate and identify all the shipping containers and open parking spots (see e.g. [0020]).
Re-claim 6, Grundfest et al. teach -the system for visualizing inventory of the intermodal container yard of Claim 1, wherein the visual representation of each shipping container physically located in the intermodal container yard is based on a carrier identification of the particular shipping container.
(see e.g. [0039] A smart container yard information system 113 maintains a representation of the current state of a smart container yard 100 with a degree of precision and completeness that enables safe operation of the vehicles and equipment in the container yard 100. The informational system 113 may furthermore maintain, or receive in real-time, operational signals representative of an operational plan for carrying out operations in the container yard 100. For example, the operational plan may determine an identifier a carrier truck 103 at the entry gate 110 (or other vehicle in the container yard 100) and perform a lookup of the identifier in a vehicle database to obtain the operational plan. )
Re-claim 7, Grundfest et al. teach --the system for visualizing inventory of the intermodal container yard of Claim 1, wherein the vehicle comprises:
a container delivery vehicle;
an aerial vehicle;
an automobile;
a truck;
a golf cart;
an all-terrain vehicle (ATV);
an autonomous vehicle;
a motorcycle; or
a remote-control vehicle.
(see e.g. [0053] For instance, an aerial drone 116 accompanying a yard truck 115 may be forced to hold its position
[0077] In an embodiment, a driver of a carrier truck 103 and any accompanying personnel may be transported between staging areas attached to gates 110, 111 by autonomous ground vehicles.
[0037] Alternatively, the carrier truck 103 may comprise an autonomous or semi-autonomous vehicle that includes an autonomous driving system that automatically controls navigation responsive to sensed environment conditions.
[0040] In an embodiment, the information system 113 may use manual modelling performed according to technical specifications, analysis of measurements acquired by sensor arrays 107 or imported vendor datasets, or other sources to produce physical models of ground vehicles (such as forklifts 114 and yard trucks 115) and aerial drones 116 servicing the container yard 100 as well as of visiting trailer trucks 103.)
Claims 8, 15 recite similar limitations as claim 1 and are therefore rejected under the same arts and rationale.
Claims 9, 16 recite similar limitations as claim 2 and are therefore rejected under the same arts and rationale.
Claims 10, 17 recite similar limitations as claim 3 and are therefore rejected under the same arts and rationale.
Claims 11, 18 recite similar limitations as claim 4 and are therefore rejected under the same arts and rationale.
Claims 13, 20 recite similar limitations as claim 6 and are therefore rejected under the same arts and rationale.
Claim 14 recites similar limitations as claim 7 and is therefore rejected under the same arts and rationale.
Claims 5, 12, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Grundfest et al. (US 20200254619 A1), in view of ROSAS-MAXEMIN et al. (WO 2023107584), in view of Strickland et al. (US 5984503 A), in further view of TIAN et al. (CN 113496563 A).
Re-claim 5, Grundfest et al., teach the metadata .
(see e.g. [0105] In an embodiment, the onboard computing system of a utility vehicle such as a forklift 114 or a container handler may transmit the commencement timestamp of extension or other operations changing the shape of a utility vehicle and metadata describing the process such as extension speed and the expected extension length to the container yard information system 113.
[0045] In an embodiment, the information system 113 may apply computer vision methods to read a license plate of the carrier truck 103 carrying a container and subsequently query a database of vehicles allowed entry to the container yard 100 for the license plate number parsed in order to perform container identification.
[0105] In an embodiment, the onboard computing system of a utility vehicle such as a forklift 114 or a container handler may transmit the commencement timestamp of extension or other operations changing the shape of a utility vehicle and metadata describing the process such as extension speed and the expected extension length to the container yard information system 113.)
Grundfest et al., in view of ROSAS-MAXEMIN et al., in view of Strickland et al., do not teach the following limitation as claimed.
However, TIAN et al. et al. teach -- The system for visualizing inventory of the intermodal container yard of Claim 2, wherein the metadata about the particular shipping container further comprises a confidence value associated with the current carrier identification of the particular shipping container.
(see e.g. TIAN et al. -Exemplary, when determining the target container image from the multi-frame container image, the terminal device can determine the confidence of the container identification in each frame of container image through the neural network, and according to the confidence of the container identification in each frame of container image; A target container image is determined from a multi-frame container image. For example, the container image of the maximum confidence of the container identification, determining as the target container image. or determining part of container image (such as odd frame image in all container images, or even number frame image in all container images) by the neural network; and determining the target container image from the multi-frame container image according to the confidence degree of the container identifiers. For example, the container image of the maximum confidence of the container identification, determining as the target container image.
Exemplary, based on confidence of each container identification, if the maximum confidence is greater than the preset threshold value, the terminal device can the container identification of the container image of the maximum confidence, determining as the target container image. or, if the maximum confidence level is not greater than the preset threshold value, the terminal device determines that all container images are not the target container image, namely cannot obtain the container identifier from the container image.)
If the container vehicle enters the smart gate according to the first acquisition time and the second acquisition time, determining the container identification of the container vehicle according to the target container image, and determining the license plate identification of the container vehicle according to the target vehicle image;
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Grundfest et al., in view of ROSAS-MAXEMIN et al., in view of Strickland et al., and include the steps cited above, as taught by TIAN et al., in order to improve the identification accuracy of container identification and license plate identification, and also determine that the container vehicle is legal container vehicle according to the container identification and the license plate identification (see e.g. TIAN et al.).
Claims 12, 19 recite similar limitations as claim 5 and is therefore rejected under the same arts and rationale.
Response to arguments
Applicant’s arguments with respect to the Non Final action dated 3/30/26 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Huat (US 20110221573 A1) -Real Time Monitoring Of Ship Cargo.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LUNA CHAMPAGNE whose telephone number is (571)272-7177. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Florian Zeender can be reached at 571 272-6790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LUNA CHAMPAGNE/Primary Examiner, Art Unit 3627 September 25, 2026
