DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-5, 8, 9, 13-15, 17, 18, and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Gong et al. (US 12,394,205).
Regarding claim 1, Gong discloses:
obtaining a plurality of multi-dimensional data including a container capable of containing an object based on time-series information (see col 6 lines 19-42 and fig 3B, continuously obtaining RGB images and corresponding depth data of a food container containing food);
obtaining first reference time multi-dimensional data based on whether the plurality of multi-dimensional data are changed (see 6:27-31 and 8:53-67, obtaining first data at a first timing determined according to changes detected in the RGB images);
identifying whether a plurality of additional multi-dimensional data after the plurality of multi-dimensional data are changed (see 6:27-31 and 8:53-67, identifying changes in additional RGB images by continuous monitoring);
obtaining second reference time multi-dimensional data among the plurality of additional multi-dimensional data based on identified change (see 6:27-31 and 8:53-67, obtaining second data at a second timing determined according to changes detected in the additional RGB images); and
predicting the volume of the object based on the first reference time multi- dimensional data and the second reference time multi-dimensional data (see 8:4-16, tracking the volume of the food in the container based on the first and second data).
Regarding claim 2, Gong further disclose: wherein the plurality of multi-dimensional data including the container capable of containing the object is obtained based on a process of obtaining a plurality of depth data including the container capable of containing the object by using a depth camera (see rejection of claim 1, depth data of the food container containing food is obtained by a depth camera).
Regarding claim 3, Gong further discloses: wherein the multi-dimensional data includes RGBD data (see rejection of claim 1, RGB and depth).
Regarding claim 4, Gong further discloses: wherein the obtaining of the plurality of multi-dimensional data including the container capable of containing the object based on the time-series information (see rejection of claim 1, continuously obtaining the RGB images and corresponding depth data of the food container containing the food) further includes:
identifying an area including the container for at least one of the plurality of obtained multi-dimensional data (see 6:55-67, identifying an area including the container in the RGB images and corresponding depth data); and
filtering the multi-dimensional data based on the identified area (see 6:55-67, segmenting the area).
Regarding claim 5, Gong further discloses: wherein the filtering of the multi-dimensional data based on the identified area includes: filtering the multi-dimensional data based on depth data of the identified area (see 6:55-67, segmenting the area includes segmenting the depth data of the area).
Regarding claim 8, Gong further discloses: wherein the obtaining of the first reference time multi-dimensional data based on whether there are the changes between the plurality of multi-dimensional data (see rejection of claim 1, obtaining the first data at the first timing determined according to changes detected in the RGB images) includes:
identifying a first area included in first multi-dimensional data among the plurality of multi-dimensional data (see 7:16-20,44-53, identifying a first pan area in a first RGB image and corresponding depth data);
identifying whether a data value of the first area is changed in second multi- dimensional data at a time after the first multi-dimensional data among the plurality of multi-dimensional data (see 7:16-20,44-53 and 8:53-67, identifying whether pixel values of the first pan area is changed in a second RGB image); and
obtaining first reference time multi-dimensional data based on whether the data value of the first area has changed (see 8:53-67 and rejection of claim 1, obtaining the second RGB image and corresponding depth data as the first data if the pixel values of the first pan area has changed by less than a threshold amount (i.e., when there is no motion)).
Regarding claim 9, Gong further discloses: wherein the obtaining of the first reference time multi-dimensional data based on whether the data value of the first area has changed (see rejection of claim 1, obtaining the first data if the pixel values of the first pan area has changed) includes:
obtaining the second multi-dimensional data as the first reference time multi- dimensional data when the identified change of the data value of the first area is within a predetermined threshold (see rejection of claim 8, obtaining the second RGB image and corresponding depth data as the first data if the pixel values of the first pan area has changed by less than the threshold amount).
Regarding claims 13-15, 17, and 18, Gong discloses everything claimed as applied above (see rejection of claims 1, 4, 5, 8, and 9), and further discloses: a computer program stored in a non-transitory computer-readable storage medium, wherein the computer program causes one or more processors to perform operations for predicting a volume of an object when the computer program is executed by the one or more processors (see 13:33-53, a computer program).
Regarding claim 20, Gong discloses everything claimed as applied above (see rejection of claim 1), and further disclose: at least one processor; and a memory (see fig 6, a processor and a memory).
Allowable Subject Matter
Claims 6, 7, 10-12, 16, and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 6, Gong discloses everything claimed as applied above (see rejection of claim 4), however, does not disclose: extracting an external contour of an object for at least one of the plurality of obtained multi-dimensional data; extracting an internal contour of the container for at least one of the plurality of obtained multi-dimensional data; and identifying an area including the container for at least one of the plurality of obtained multi-dimensional data based on the external contour of the object and the extracted internal contour of the container. Similar reasons apply to claim 16.
Regarding claim 10, Gong discloses everything claimed as applied above (see rejection of claim 1), however, does not disclose: identifying a change in data value between first additional multi-dimensional data among the plurality of additional multi-dimensional data, and the first reference time multi-dimensional data; and when the identified change in data value between the first additional multi- dimensional data and the first reference time multi-dimensional data is equal to or larger than a predetermined threshold, comparing the first additional multi-dimensional data and the first reference time multi-dimensional data to determine that the change of the data value occurs. Similar reasons apply to claim 19.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Connor (US 2023/0335253), Pi (US 2018/0149583), Kim (US 2022/0359058), and Kim (US 2020/0365250) each discloses predicting the volume of food consumed using depth information. Erbe et al. (US 2017/0115008) discloses predicting the volume of food using projection light for measuring varying height of food.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SJ PARK whose telephone number is (571)270-3569. The examiner can normally be reached M-F 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, EMILY TERRELL can be reached at 571-270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SJ Park/Primary Examiner, Art Unit 2675
