DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
1. The specification is objected to as failing to provide proper antecedent basis for the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the following is required:
Each of claims 3, 10, and 17 recite “wherein the AI filtering and labeling techniques are selected from a list consisting of: convolutional neural networks, region-based convolutional neural networks, and you only look once algorithms.”, however, the Examiner was unable find support for this limitation in the Specification. The claim language presented reads as if the method chooses one of the different types of algorithms from a list. However, the Examiner was unable to find support in the Specification dated 26 June, 2023. The Examiner did find support for the multiple types of algorithms being presented as various options in P[0038]: “Here, the hygiene assessment program 108 may analyze the image to identify common objects in an image using an AI classification algorithm. The AI classification algorithm may utilize any technique to identify common objects within the image. For example, hygiene assessment program 108 may employ one or more convolutional neural networks (CNN), which are deep learning algorithms that can be used for image classification; through edge detection, a CNN can identify shapes in an image. The hygiene assessment program 108 may additionally or alternatively utilize region-based convolutional neural networks (R-CNN), which constitute an extension of the CNN which may be used for object detection in an image; R-CNN first identifies potential regions of interest, and then uses CNN to classify the objects in those regions. The hygiene assessment program 108 may use the You Only Look Once (YOLO) technique, which is a real-time object detection algorithm that can be used for both image classification and object detection.”. Therefore, the Examiner will treat claims 3, 10, and 17 as multiple options for the algorithm linked by the exclusive “or” where only one option of the several provided needs to be present for the claim to be satisfied.
Each of claims 3, 10, and 17 recite “normalizing the classifying”, which is not clearly understood by the Examiner; However, the Examiner was unable find support for this limitation in the Specification. In the provided Specification, the Examiner points to P[0047]: “In embodiments, the hygiene assessment program 108 may normalize a hygiene result”, which does not provide support for “normalizing the classifying”.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
2. Claims 3, 5, 10, 12, 17 and 19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Each of claims 3, 10, and 17 recite “wherein the AI filtering and labeling techniques are selected from a list consisting of: convolutional neural networks, region-based convolutional neural networks, and you only look once algorithms.”, however, the Examiner was unable find support for this limitation in the Specification. The claim language presented reads as if the method chooses one of the different types of algorithms from a list. However, the Examiner was unable to find support in the Specification dated 26 June, 2023. The Examiner did find support for the multiple types of algorithms being presented as various options in P[0038]: “Here, the hygiene assessment program 108 may analyze the image to identify common objects in an image using an AI classification algorithm. The AI classification algorithm may utilize any technique to identify common objects within the image. For example, hygiene assessment program 108 may employ one or more convolutional neural networks (CNN), which are deep learning algorithms that can be used for image classification; through edge detection, a CNN can identify shapes in an image. The hygiene assessment program 108 may additionally or alternatively utilize region-based convolutional neural networks (R-CNN), which constitute an extension of the CNN which may be used for object detection in an image; R-CNN first identifies potential regions of interest, and then uses CNN to classify the objects in those regions. The hygiene assessment program 108 may use the You Only Look Once (YOLO) technique, which is a real-time object detection algorithm that can be used for both image classification and object detection.”. Therefore, the Examiner will treat claims 3, 10, and 17 as multiple options for the algorithm linked by the exclusive “or” where only one option of the several provided needs to be present for the claim to be satisfied.
Each of claims 3, 10, and 17 recite “normalizing the classifying”, which is not clearly understood by the Examiner; However, the Examiner was unable find support for this limitation in the Specification. In the provided Specification, the Examiner points to P[0047]: “In embodiments, the hygiene assessment program 108 may normalize a hygiene result”, which does not provide support for “normalizing the classifying”, and instead provides support for normalizing a hygiene result.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
3. Claims 1-6, 8-13, and 15-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US 20210027485 A1: Le Zhang, (herein after “Zhang”).
Regarding claim 1, A processor-implemented method (Zhang, P[0165]: “The term “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers.”) for classifying images (Zhang, P[0003]: “the system can detect areas such as floors and counters and classify portions of the areas as being clean or dirty.”), the method comprising:
receiving an image of a critical surface of a plurality of critical surfaces comprising a location (Zhang, P[0003]: “the system can detect areas such as floors and counters and classify portions of the areas as being clean or dirty.”, where the critical surfaces are the tables, as shown in Fig. 2.);
classifying the critical surface as clean or unclean based on a plurality of AI image filtering and labeling techniques (Zhang, P[0003]: “the system can detect areas such as floors and counters and classify portions of the areas as being clean or dirty.”, where the critical surfaces are the tables, as shown in Fig. 2., and P[0068]: “The system 100 provides uses artificial intelligence and machine vision to detect various conditions and issues. The system 100 can operate using data from cameras placed to capture images of public areas, such as retail stores, restaurants, and so on”); and
uploading the image and the classification to a remotely accessible digital repository (Zhang, P[0165]: “Embodiments of the invention can be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them.”, and in P[0075]: “a remote computer system 130 … The various devices communicate over a network”, where the remote computer system contains the remote memory for storing images with their classification as shown in Fig. 2.).
Regarding claim 2, wherein the image is received from a static or mobile vehicle mounted camera observing the critical surface (Zhang, Fig. 1A shows the camera 110a, and Fig. 2 shows the view of the camera covering the critical surfaces.).
Regarding claim 3, as best understood, wherein the AI filtering and labeling techniques are selected from a list consisting of:
convolutional neural networks (Zhang, P[0082]: “the models 123 that process image data can use the faster R-CNN object detection and recognition framework.”), region-based convolutional neural networks (Zhang, P[0082]: “the models 123 that process image data can use the faster R-CNN object detection and recognition framework”, which is region based, and “To improve efficiency, the region proposal network (RPN) can be arranged to share full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals. An RPN can be a fully convolutional network that simultaneously predicts object bounds”), and you only look once algorithms.
Regarding claim 4, wherein the classifying is further based on identifying one or more anomalies on the critical surface using one or more anomaly detection methods (Zhang, P[0003]: “the system can detect areas such as floors and counters and classify portions of the areas as being clean or dirty.”, where the critical surfaces are the tables and the anomalies are the dirty portions, as shown in Fig. 2.).
Regarding claim 5, as best understood, wherein the anomaly detection methods comprise Anomaly Detection in Time Series Images (Zhang, P[0009]: “The system can keep track of the state of conditions that it has detected. For example, track the progress of detected issues over time, determining whether problematic conditions remain over time and reminding and escalating notifications to appropriate systems or users to ensure that conditions that need attention are addressed and not forgotten. The system can detect when conditions are changed or when issues are resolved, such as when a dirty table is cleaned”), and wherein the method further comprises:
normalizing the classifying to account for a duration that a detected anomaly has persisted on the critical surface (Zhang, P[0009]: “The system can keep track of the state of conditions that it has detected. For example, track the progress of detected issues over time, determining whether problematic conditions remain over time and reminding and escalating notifications to appropriate systems or users to ensure that conditions that need attention are addressed and not forgotten. The system can detect when conditions are changed or when issues are resolved, such as when a dirty table is cleaned, when a low-stocked item is replenished, or more generally when a monitored area returns to one of a set of desirable states or conditions. If the system determines that a condition that deviates from the range of proper operating conditions for the monitored area persists for a certain amount of time, such as 30 minutes after notifying a worker involved, the system can take additional actions, such as sending a reminder, informing a supervisor, assigning an additional worker to address the condition, etc.”).
Regarding claim 6, further comprising: responsive to determining that at least one image has been uploaded for each of the plurality of critical surfaces (Zhang, §Abstract: “Input data based on the image data is provided to one or more machine learning models trained to detect different properties of the monitored area.”, and Fig. 2, which shows the image of the plurality of critical surfaces that is uploaded.), uploading a hygeine report to the digital repository (Zhang, P[0071]: “The system 100 can be configured to provide alerts and analytics to provide better control to managers and owners … Real-time reports, daily emails, and operations benchmarks-”, and P[0009]: “The system can keep track of the state of conditions that it has detected. For example, track the progress of detected issues over time, determining whether problematic conditions remain over time and reminding and escalating notifications to appropriate systems or users to ensure that conditions that need attention are addressed and not forgotten. The system can detect when conditions are changed or when issues are resolved, such as when a dirty table is cleaned-”
Claims 8 and 15 recite features nearly identical to those recited in claim 1. Claims 8 and 15 are rejected for reasons analogous to those discussed above in conjunction with claim 1.
Claims 9 and 16 recite features nearly identical to those recited in claim 2. Claims 9 and 16 are rejected for reasons analogous to those discussed above in conjunction with claim 2.
Claims 10 and 17 recite features nearly identical to those recited in claim 3. Claims 10 and 17 are rejected for reasons analogous to those discussed above in conjunction with claim 3.
Claims 11 and 18 recite features nearly identical to those recited in claim 4. Claims 11 and 18 are rejected for reasons analogous to those discussed above in conjunction with claim 4.
Claims 12 and 19 recite features nearly identical to those recited in claim 5. Claims 12 and 19 are rejected for reasons analogous to those discussed above in conjunction with claim 5.
Claims 13 and 20 recite features nearly identical to those recited in claim 6. Claims 13 and 20 are rejected for reasons analogous to those discussed above in conjunction with claim 6.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of the Wikipedia Article for Blockchain.
Regarding claim 7, Zhang discloses in P[0165]: “Embodiments of the invention and all of the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the invention can be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them.”, but Zhang does not discloses that their digital memory is specifically “blockchain”, and therefore does not explicitly disclose “wherein the digital repository comprises a blockchain.”
However, the Wikipedia Article for Blockchain discloses the uses for blockchains as digital repositories.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang to include utilizing blockchain as a digital repository/memory, as taught by the Wikipedia article for Blockchain, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have provided the benefit of reducing risks that come with centrally held data.
Claim 14 recites features nearly identical to those recited in claim 7. Claim 14 is rejected for reasons analogous to those discussed above in conjunction with claim 7.
Conclusion
5. Any inquiry concerning this communication or earlier communications from the examiner should be directed to TY M BEATTY whose telephone number is (703)756-5370. The examiner can normally be reached Mon-Fri: 8AM-4PM EST..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571) - 272 - 3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TY MITCHELL BEATTY/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights