DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see Applicants Remarks pages 1-3, filed 02/23/26, with respect to the rejection(s) of claim(s) 1, 2, 4-13, 14-22 have been fully considered and is not persuasive.
Regarding claim 1, Applicant states that the prior art reference Magal fails to teach when the subset includes at least a threshold number of images indicative of the determined load bay state, completing the subset (Applicant Remarks pages 2-3). Examiner disagrees with Applicant. Magal teaches in response to a detected status event, store in a memory, 2D image data captured prior to the status event, and 2D image data captured after the status event. For example, when an invalid door status is detected or a safety violation is detected by evaluation of the 2D image data, the evaluation module may store relevant 2D image data. the relevant 2D image data may be image data that is collected during a time span before and after the detection of the door status where the range of the time span may be predetermined. To collect this window of image data, the TMU may be configured to buffer a pre-determined amount of captured 2D image data during image capture processes. see 0059)). This would read on when the subset includes at least a threshold number of images indicative of the determined load bay state, completing the subset.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 8-17, 19-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Magal ([USAP 2020/0014888)) in view of Vijayanarasimhan US 9076065.
Referring to claim 1, Magal teaches a method ([a method and system as shown in fig 1-5]), comprising: at a load bay, ([as shown in fig 1, a docking bay (e.g., docking bays 102d-110d) of loading facility 101.a loading dock 100 of fig 1, including a loading facility, see 0014, a plurality of docking bays, a plurality of vehicles, and a plurality of vehicle storage areas] see 0003]), controlling an imaging device disposed at the load bay to capture a plurality of images of the load bay; ([0017] FIG. 2 also depicts a trailer may be a mountable device that includes a 3D- depth camera for capturing 3D images (e.g., 3D image data) and a photo-realistic camera (e.g., 2D image data see 0054]),
at a computing device communicatively coupled to the imaging device, (see 0031, [one or more communication protocol standards including, for example, TCP/IP, WiFi (802.11b), Bluetooth, or any other similar communication protocols or standards, to communicate with the TMU 202 may process the 3D and 2D image data, as scanned or sensed from the 3D-depth camera and photo-realistic camera, for use by other device, see 0018));
obtaining a subset of the plurality of images; ([the TMU 202 may process the 3D and 2D image data, as scanned or sensed from the 3D-depth camera and photo- realistic camera] see 0018));
obtaining an image classification for each image in the subset; ([the convolutional neural network analyzes the RGB image, and obtains an output of encoded labels based on the analysis of the RGB image, the neural network is trained to appropriately classify the images collected over a period of time and output an encoded vector that indicates the different labels that are attributed to the image, see 0045]); and
determining a load bay state based on the representative class for the subset, ([status data, or alert data as determined from the original scanned or sensed image data, see 00180) see also [the evaluation module may also be configured to send status events to other components in the system 300, for example, reporting a door close, door open, or alert event to the server 302 that may be processed by a user of the client device 204, see 0034-0035)).
wherein obtaining the subset comprises: in response to capturing a subsequent image, if the subsequent image is captured within a threshold time of a preceding image in the subset, adding the subsequent image to the subset (in response to a detected status event, store in a memory, 2D image data captured prior to the status event, and 2D image data captured after the status event. For example, when an invalid door status is detected or a safety violation is detected by evaluation of the 2D image data, the evaluation module may store relevant 2D image data. the relevant 2D image data may be image data that is collected during a time span before and after the detection of the door status where the range of the time span may be predetermined (paragraph 0059); and when the subset includes at least a threshold number of images indicative of the determined load bay state, completing the subset (To collect this window of image data, the TMU may be configured to buffer a pre-determined amount of captured 2D image data during image capture processes. see 0059)).
Although Magal teaches an output of [0.9, 0.1, 0.2, 0.15] represents the confidence scores (represented as probabilities) of the convolutional neural network that it has assigned to each of the categories where the input image falls into. Here the neural network is 90% confident that the door is closed and since the closed and open state are mutually exclusive the network assigns the remaining probability to the open category (paragraph 0030);
Magal fails to teach identifying a representative class for the subset based on a weighted confidence of the image classification for each image in the subset
Vijayanarasimhan teaches identifying a representative class for the subset based on a weighted confidence of the image classification for each image in the subset (The confidence level or score can be different from or the same as the score (e.g., weighted sum or score associated therewith) used to determine whether an object instance of an object of interest is identified and/or classify whether a cluster of filter activations represents or identifies the object of interest. For instance, for a particular visual image, the classifier component 204 may determine that the weighted sum of activation scores of the filter activations for a particular cluster has a value that indicates an instance of the object of interest in a particular location of the visual image. However, upon evaluation of results of the evaluation or classification of the visual image by the classifier component 204, the trainer component 204 may determine that the confidence level in those results is relatively low (e.g., below a defined threshold confidence level for automatically adding information relating to the visual image to the subset of training examples 210) (column 9, lines 36-52)
Therefore, it would have been obvious to a person with ordinary skill in the art to have modified Magal to include: identifying a representative class for the subset based on a weighted confidence of the image classification for each image in the subset.
The reason of doing so is to accurately identify and label a classification of an image.
Referring to claim 2, Magal teaches a method ([a method and system as shown in fig 1-5]), comprising, wherein each image in the subset is captured ([the TMU 202 may process the 3D and 2D image data, as scanned or sensed from the 3D-depth camera and photo-realistic camera] see 0018]), within a threshold time of a preceding image in the subset, ([see 0030, set a threshold for all confidence scores and any score for a label exceeding the threshold is output to the server for alert generation)), ([with in a camera, in general each image must be taken within a certain predefined time period after the previous image in that sequence, essentially describing a requirement for capturing images rapidly in succession, with a set time limit between each capture image, so it is inherent imaging camera has this function, so it is inherent]).
Referring to claim 4, Magal teaches a method ([a method and system as shown in fig 1-5]), comprising, further comprising: if the subsequent image is not captured within the threshold time of the preceding image in the subset, discarding the subset and generating a new subset with the subsequent image, ([the TMU 202 may process the 3D and 2D image data, as scanned or sensed from the 3D-depth camera and photo-realistic camera] see 0018]), within a threshold time of a preceding image in the subset, ([see 0030, set a threshold for all confidence scores and any score for a label exceeding the threshold is output to the server for alert generation]), (in a camera, in general each image must be taken within a certain predefined time period after the previous image in that sequence, essentially describing a requirement for capturing images rapidly in succession, with a set time limit between each capture image, so it is inherent imaging having this function is common, so it is inherent]).
Referring to claim 5, Magal teaches a method ([a method and system as shown in fig 1-5]), further comprising: obtaining the image classification of the subsequent image ([FIG. 3. the convolutional neural network analyzes the RGB image, and obtains an output of encoded labels based on the analysis of the RGB image, see 0045, the neural network is trained to appropriately classify the images collected over a period of time and output an encoded vector that indicates the different labels that are attributed to the image]); and if a confidence level for the image classification of the subsequent image is below a threshold confidence level, discarding the subsequent image, ([the systems and methods disclosed herein may set a threshold for all confidence scores and any score for a label exceeding the threshold is output to the server for alert generation, see 0032]).
Referring to claim 6, Magal teaches a method ([a method and system as shown in fig 1-5]), wherein obtaining the image classification comprises processing the image by a machine learning-based image classifier, ([the neural network is trained to appropriately classify the images collected over a period of time and output an encoded vector that indicates the different labels that are attributed to the image, see 0045)).
Referring to claim 8, Magal teaches a method, ([a method and system as shown in fig 1-5]), wherein identifying a representative class for the subset comprises selecting the image classification of the images in the subset having a largest weighted confidence, ([the systems and methods disclosed herein may set a threshold for all confidence scores and any score for a label exceeding the threshold is output to the server for alert generation, see 0030-0032)).
Referring to claim 9, Magal teaches a method ([a method and system as shown in fig 1-5]),wherein the load bay state comprises a dock state. a trailer door state and a load parameter, ([0054] FIG. 7 is a flow chart of a method 700 for detecting the status of a door, see also ([0013] the systems and methods described herein may be useful at loading docks for detecting when trailer doors are open or closed. FIG. 1 is a perspective view, as seen from above, of a loading dock 100 including a loading facility 101)).
Referring to claim 11, Magal teaches a method ([a method and system as shown in fig 1-5]), further comprising transmitting the load bay state to a further computing device for output at the further computing device, ([use of existing RGB camera's continuous image frames are sent as inputs to a trained convolutional neural network image classifier to detect real-time status of a trailer door to determine whether the door is in an open or closed status, see 0043)).
Referring to claim 12, Magal teaches a system ([a method and system as shown in fig 1-5]), comprising: at a load bay, ([as shown in fig 1, a docking bay (e.g., docking bays 102d-110d) of loading facility 101.a loading dock 100 of fig 1, including a loading facility, see 0014, a plurality of docking bays, a plurality of vehicles, and a plurality of vehicle storage areas] see 0003]), controlling an imaging device disposed at the load bay to capture a plurality of images of the load bay; ([0017] FIG. 2 also depicts a trailer monitoring unit (TMU) 202. TMU 202 may be a mountable device that includes a 3D-depth camera for capturing 3D images (e.g., 3D image data) and a photo-realistic camera (e.g., 2D image data). ([TMU 202 of FIGS. 2 and 2B, the one or more processors may be processors(s) (e.g., processor 306) of the server 304 of fig 7, see 0055]), imaging camera ([a 2D camera, (e.g. the 2D camera 316 of TMU 202) captures 2D image data] see 0054]), at a computing device communicatively coupled to the imaging device, (see 0031, [one or more communication protocol standards including, for example, TCP/IP, (802.11b), Bluetooth, or any other similar communication protocols or standards, to communicate with the TMU 202 may process the 3D and 2D image data, as scanned or sensed from the 3D-depth camera and photo-realistic camera, for use by other device, see 0018));
obtaining a subset of the plurality of images; ([the TMU 202 may process the 3D and 2D image data, as scanned or sensed from the 3D-depth camera and photo- realistic camera] see 0018));
obtaining an image classification for each image in the subset; ([the convolutional neural network analyzes the RGB image, and obtains an output of encoded labels based on the analysis of the RGB image, the neural network is trained to appropriately classify the images collected over a period of time and output an encoded vector that indicates the different labels that are attributed to the image, see 0045]);
determine a load bay state based on the representative class for the subset, ([status data, or alert data as determined from the original scanned or sensed image data, see 00180) see also [the evaluation module may also be configured to send status events to other components in the system 300, for example, reporting a door close, door open, or alert event to the server 302 that may be processed by a user of the client device 204, see 0034-0035)).
wherein to obtain the subset, the computing device is configured to: in response to capturing a subsequent image, if the subsequent image is captured within a threshold time of a preceding image in the subset, add the subsequent image to the subset(in response to a detected status event, store in a memory, 2D image data captured prior to the status event, and 2D image data captured after the status event. For example, when an invalid door status is detected or a safety violation is detected by evaluation of the 2D image data, the evaluation module may store relevant 2D image data. the relevant 2D image data may be image data that is collected during a time span before and after the detection of the door status where the range of the time span may be predetermined (paragraph 0059); and when the subset includes at least a threshold number of images indicative of the determined load bay state, completing the subset (To collect this window of image data, the TMU may be configured to buffer a pre-determined amount of captured 2D image data during image capture processes. see 0059)).
Although Magal teaches an output of [0.9, 0.1, 0.2, 0.15] represents the confidence scores (represented as probabilities) of the convolutional neural network that it has assigned to each of the categories where the input image falls into. Here the neural network is 90% confident that the door is closed and since the closed and open state are mutually exclusive the network assigns the remaining probability to the open category (paragraph 0030);
Magal fails to teach identifying a representative class for the subset based on a weighted confidence of the image classification for each image in the subset
Vijayanarasimhan teaches identify a representative class for the subset based on a weighted confidence of the image classification for each image in the subset (The confidence level or score can be different from or the same as the score (e.g., weighted sum or score associated therewith) used to determine whether an object instance of an object of interest is identified and/or classify whether a cluster of filter activations represents or identifies the object of interest. For instance, for a particular visual image, the classifier component 204 may determine that the weighted sum of activation scores of the filter activations for a particular cluster has a value that indicates an instance of the object of interest in a particular location of the visual image. However, upon evaluation of results of the evaluation or classification of the visual image by the classifier component 204, the trainer component 204 may determine that the confidence level in those results is relatively low (e.g., below a defined threshold confidence level for automatically adding information relating to the visual image to the subset of training examples 210) (column 9, lines 36-52)
Therefore, it would have been obvious to a person with ordinary skill in the art to have modified Magal to include: identifying a representative class for the subset based on a weighted confidence of the image classification for each image in the subset.
The reason of doing so is to accurately identify and label a classification of an image.
Referring to claim 13, Magal teaches a system ([a method and system as shown in fig 1-5]), comprising, wherein each image in the subset is captured ([the TMU 202 may process the 3D and 2D image data, as scanned or sensed from the 3D-depth camera and photo-realistic camera] see 0018]), within a threshold time of a preceding image in the subset, ([see 0030, set a threshold for all confidence scores and any score for a label exceeding the threshold is output to the server for alert generation]),
((the relevant 2D image data may be image data that is collected during a time span before and after the detection of the door status where the range of the time span may be predetermined. see 0059)) [with in a camera, in general each image must be taken within a certain predefined time period after the previous image in that sequence, essentially describing a requirement for capturing images rapidly in succession, with a set time limit between each capture image, so it is inherent imaging camera has this function, so it is inherent]).
Referring to claim 15, Magal teaches a system ([a method and system as shown in fig 1-5]), comprising, further comprising: if the subsequent image is not captured within the threshold time of the preceding image in the subset, discarding the subset and generating a new subset with the subsequent image, ([the TMU 202 may process the 3D and 2D image data, as scanned or sensed from the 3D-depth camera and photo-realistic camera] see 0018]), within a threshold time of a preceding image in the subset, ((the relevant 2D image data may be image data that is collected during a time span before and after the detection of the door status where the range of the time span may be predetermined. see 0059) [see 0030, set a threshold for all confidence scores and any score for a label exceeding the threshold is output to the server for alert generation]), (in a camera, in general each image must be taken within a certain predefined time period after the previous image in that sequence, essentially describing a requirement for capturing images rapidly in succession, with a set time limit between each capture image, so it is inherent imaging having this function is common, so it is inherent)).
Referring to claim 16, Magal teaches a system ([a method and system as shown in fig 1-5]), further comprising: obtaining the image classification of the subsequent image ([FIG. 3. the convolutional neural network analyzes the RGB image, and obtains an output of encoded labels based on the analysis of the RGB image, see 0045, the neural network is trained to appropriately classify the images collected over a period of time and output an encoded vector that indicates the different labels that are attributed to the image]); and if a confidence level for the image classification of the subsequent image is below a threshold confidence level, discarding the subsequent image, ([the systems and methods disclosed herein may set a threshold for all confidence scores and any score for a label exceeding the threshold is output to the server for alert generation, see 0032]).
Referring to claim 17, Magal teaches a system ([a method and system as shown in fig 1-5]), wherein obtaining the image classification comprises processing the image by a machine learning-based image classifier, ([the neural network is trained to appropriately classify the images collected over a period of time and output an encoded vector that indicates the different labels that are attributed to the image, see 0045)).
Referring to claim 19, Magal teaches a system, ([a method and system as shown in fig 1-5]), wherein identifying a representative class for the subset comprises selecting the image classification of the images in the subset having a largest weighted confidence, ([the systems and methods disclosed herein may set a threshold for all confidence scores and any score for a label exceeding the threshold is output to the server for alert generation, see 0032).
Referring to claim 20, Magal teaches a system ([a method and system as shown in fig 1-5]),wherein the load bay state comprises a dock state. a trailer door state and a load parameter, ([0054] FIG. 7 is a flow chart of a method 700 for detecting the status of a door, see also ([0013] the systems and methods described herein may be useful at loading docks for detecting when trailer doors are open or closed. FIG. 1 is a perspective view, as seen from above, of aloading dock 100 including a loading facility 101)).
Referring to claim 22, Magal teaches a system ([a method and system as shown in fig 1-5]), further comprising transmitting the load bay state to a further computing device for output at the further computing device, ([use of existing RGB camera's continuous image frames are sent as inputs to a trained convolutional neural network image classifier to detect real-time status of a trailer door to determine whether the door is in an open or closed status, see 0043]).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL L BURLESON whose telephone number is (571)272-7460. The examiner can normally be reached 9am to 530pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi Sarpong can be reached on (571) 270- 3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Michael Burleson
Patent Examiner
Art Unit 2683
Michael Burleson
June 3, 2026
/MICHAEL BURLESON/
/AKWASI M SARPONG/SPE, Art Unit 2681 6/8/2026
Prosecution Insights