DETAILED ACTION
This final Office action is responsive to amendments filed April 22nd, 2026. Claims 1 and 9 have been amended. Claims 1-10 are presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Response to Arguments
Applicant’s arguments, see page 6, filed 04/22/26, with respect to claim 9 have been fully considered and are persuasive. The claim interpretation under 35 USC 112(f) of 02/03/26 has been withdrawn.
Applicant's arguments claim rejections under 35 USC 101 filed 04/22/26 have been fully considered but they are not persuasive.
On pages 7-28 of the provided remarks, Applicant argues that the amended claims present statutory subject matter. Beginning with Step 2A Prong 1 analysis, Applicant argues on page 8 of the provided remarks that “the combination of features of amended independent claim 1 do not cover any features that recite a mental process.” Examiner respectfully disagrees and asserts that “extracting a feature from the data of the task sound by converting the task sound into the feature for model processing” could be performed as an observation, judgement, evaluation, and opinion of the human mind. The claimed, “estimating, based on the feature” is a further evaluation and judgement of the human mind. Applicant’s arguments are not persuasive.
Continuing on pages 8-9 of the provided remarks, Applicant cites the 2019 101 Guidelines and the USPTO Memorandum dated August 4th, 2025, to argue “the subject features recite in both the previous form of claim 1, and especially as amended in this paper, could not practically be performed in the human mind without the associated hardware and the assistance of a special purpose computer programmed to apply the specialized algorithms disclosed in the specification of the present application and recited in the claims.” Examiner respectfully disagrees and asserts that the claims recitation of “extracting a feature from the data of the task sound by converting the task sound into the feature of model processing” as claimed does not require the argued “special purpose computer programmed to apply the specialized algorithms”. The claims recitation of “performed by a computer” are recited so generically (no details whatsoever are provided other than that they are general purpose computing components and regular office supplies) that they represent no more than mere instructions to apply the judicial exception on a computer. These limitations can also be viewed as nothing more than an attempt to generally link the use of the judicial exception to the technological environment of a computer. Applicant’s arguments are not persuasive.
Continuing on page 10 of the provided remarks, Applicant cites the first full paragraph of page 20 of the specification for Specification support of the argued “process and algorithms disclosed at these portions of the specification of the present application, including the detailed process flow and determinations shown therein, that such analyses would require the aid of a special purpose computer programmed to apply the specialized algorithms disclosed in the specification of the present application.” Per MPEP 2106.04(a)(III)(C) “In evaluating whether a claim that requires a computer recites a mental process, examiners should carefully consider the broadest reasonable interpretation of the claim in light of the specification. For instance, examiners should review the specification to determine if the claimed invention is described as a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept. In these situations, the claim is considered to recite a mental process.” Examiner asserts that the cited Specification for the argued “extracting” is using a computer as a tool to perform the concept of extracting a feature of image data and calculating a similarity between the extracted feature of an image in which a worker performing a task in which a transparent object appears. Applicant’s arguments are not persuasive.
On pages 10-11 of the provided remarks, Applicant cited the first two full paragraphs of page 48 of the as-filed Specification to argue that the “newly-amended independent claim 1 now more precisely and clearly recites concrete data processing of extracting a feature from task sound and estimating the extracted feature using a trained model.” Examiner respectfully disagrees and asserts that the claimed “extracting a feature from the data of the task sound by converting the task sound into the feature of model processing” does not recite the argued “concrete data processing” but a high-level observation, judgement, evaluation, and opinion of the human mind. The claimed, “estimating, based on the feature” is a further evaluation and judgement of the human mind. The cited Specification further recites “general-purpose computer” operations, which per MPEP 2106.04(a)(III)(C), “merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept. In these situations, the claim is considered to recite a mental process.” Applicant’s arguments are not persuasive.
On pages 11-13 of the provided remarks, Applicant cites the Decision on Appeal by the USPTO’s Patent Trial and Appeal Board in Appeal No. 2017-010856 for U.S Patent Application No. 14/062,126, and argues “the Examiner of the instant application has not given appropriate consideration to the claim language as a whole”. Examiner begins by asserting that the cited application and analysis regarding the practical application and improvement to technology falls under the Step 2A Prong 2 and Step 2B of the subject matter eligibility guidelines. Per MPEP 2106.04(d) and 2106.05(a) the argued “consideration to the claim language as a whole” would not render the application under Step 2A Prong 1 to not recite the abstract idea of certain methods of organizing human activity or mental process. Therefore, Applicant’s argument is moot regarding Step 2A Prong 1 analysis.
On pages 13-17 of the provided remarks, Applicant argues, citing MPEP 2106.04(a)(2)(II)(A) and portions of the as-filed Specification, “amended claim 1 of the instant application is clearly directed to improvements in an estimation device and associated methodology in which the target to be estimated is tied to the specific technical problem of a transparent-object handling task, rather than general activity estimation.” Examiner respectfully disagrees and begins by asserting, as stated above, the cited analysis regarding the practical application and improvement to technology falls under the Step 2A Prong 2 and Step 2B of the subject matter eligibility guidelines. Per MPEP 2106.04(d) and 2106.05(a) the argued “consideration to the claim language as a whole” would not render the application under Step 2A Prong 1 to not recite the abstract idea of certain methods of organizing human activity or mental process. Therefore, Applicant’s arguments are moot regarding Step 2A Prong 1. Further, per MPEP 2106.05(a), the analysis regarding improvements to technology or a technical field include the following, “An indication that the claimed invention provides an improvement can include a discussion in the specification that identifies a technical problem and explains the details of an unconventional technical solution expressed in the claim, or identifies technical improvements realized by the claim over the prior art.” Therefore, it is not simply that “the target to be estimated is tied to the specific technical problem of a transparent-object handling task, rather than general activity estimation” but that the target to be estimated involves a technical solution to a technical problem in the field of device estimation. The provided Specification support does not explicitly set forth an improvement, and Examiner cites the following per MPEP 2106.05(a), “if the specification explicitly sets forth an improvement but in a conclusory manner (i.e., a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology.” Applicant’s arguments are not persuasive.
Regarding Step 2A Prong 2 analysis, Applicant argues on page 18 of the provided remarks, “the specification describes particular advantages of the claimed inventions” and continues on to cite various portions of the Specification for support. Examiner respectfully disagrees and asserts, as stated above, the provided Specification support does not explicitly set forth an improvement, and Examiner cites the following per MPEP 2106.05(a), “if the specification explicitly sets forth an improvement but in a conclusory manner (i.e., a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology.” Further, noting various claims within the Specification that the present invention “makes it possible to accurately estimate tasks in which a transparent object is handled”, Examiner cites the following “Appellant is reminded that in most cases, “relying on a computer to perform routine tasks more quickly or more accurately is insufficient to render a claim patent eligible.” OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015); Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1370 (Fed. Cir. 2015) (“[M]erely adding computer functionality to increase the speed or efficiency of the process does not confer patent eligibility on an otherwise abstract idea.”); Alice, 573 U.S. at 223 (“Thus, if a patent’s recitation of a computer amounts to a mere instruction to implement an abstract idea on a computer, that addition cannot impart patent eligibility.”)” Applicant’s arguments are not persuasive.
Citing the USPTO Memorandum dated August 4th, 2025, Applicant argues on pages 20-21 of the provided remarks, “the claims of the present application are therefore directed to a particular, limited application of an estimation device and methodology of estimating whether a worker is performing a task in which a transparent object is handled by inputting task sound data into a trained first model.” Examiner respectfully disagrees and asserts as stated above that the amended claims do not present a particular or limiting application of the estimation device as the claimed functions are recited with a high-level of generality such that they can be performed as functions of the human mind. Applicant’s arguments are not persuasive.
Continuing on pages 22-23 of the provided remarks, Applicant argues citing MPEP 2106.05(a) and MPEP 2106.05(f) that the amended claims “recite a “technology based solution” that overcomes disadvantages of the conventional estimation devices and methodologies.” Examiner respectfully disagrees and asserts, as stated above, that the present claims do not recite a technology based solution as the claim merely recites the estimation method being “performed by a computer”. The claims recitation of “performed by a computer” are recited so generically (no details whatsoever are provided other than that they are general purpose computing components and regular office supplies) that they represent no more than mere instructions to apply the judicial exception on a computer. These limitations can also be viewed as nothing more than an attempt to generally link the use of the judicial exception to the technological environment of a computer. Applicant’s arguments are not persuasive.
On pages 23-24 of the provided remarks, Applicant cites the USPTO Memorandum dated December 5, 2025 and argues “Applicant has opted to proceed in this paper by more precisely reciting improvement associated with recite logical structures and processes, including a recitation, in newly-amended independent claim 1 of the present application, of converting the task sound into feature/representation for model processing.” Examiner begins by asserting, as stated above, that the amended recitation of “converting the task sound into feature/representation” does not present an improvement to the technical field as the high-level recitation of the conversion could be a mental judgment or evaluation of the mind. Further, the cited improvement to the operation of a machine learning model claimed in the cited Ex parte Desjardins is in no way analogous to the argued conversion of task sounds within the current application. Applicant’s arguments are not persuasive.
On pages 24-26 of the provided remarks, Applicant argues regarding Examiner’s “apply it” assertions, citing the USPTO Memorandum of August 4th, 2025, “it is evident from the foregoing remarks that the proposed amendments to independent claim 1 would result in claim 1 as a whole providing an improvement to technology or a technical field.” Examiner respectfully disagrees and asserts, citing the amended claim language verbatim, the claimed “extracting a feature from the data of the task sound by converting the task sound into the feature of model processing” does not recite the argued “concrete data processing” but a high-level observation, judgement, evaluation, and opinion of the human mind. The claimed, “estimating, based on the feature” is a further evaluation and judgement of the human mind. The cited Specification further recites “general-purpose computer” operations, which per MPEP 2106.04(a)(III)(C), “merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept. In these situations, the claim is considered to recite a mental process.” Applicant’s arguments are not persuasive.
Regarding Step 2B analysis, Applicant argues on page 26 of the provided remarks, “claim 1, especially as newly-amended, is directed to a particular, limited application of an estimation device and methodology of estimating whether a worker is performing a task in which a transparent object is handled by inputting task sound data into a trained first model”. Examiner respectfully disagrees and asserts as stated above that the amended claims do not present a particular or limiting application of the estimation device as the claimed functions are recited with a high-level of generality such that they can be performed as functions of the human mind. Further, per MPEP 2106.05(II), under Step 2B analysis, “Step 2B asks: Does the claim recite additional elements that amount to significantly more than the judicial exception? Examiners should answer this question by first identifying whether there are any additional elements (features/limitations/steps) recited in the claim beyond the judicial exception(s), and then evaluating those additional elements individually and in combination to determine whether they contribute an inventive concept (i.e., amount to significantly more than the judicial exception(s)).” Therefore, Applicant argument regarding practical application is moot.
On page 27 of the provided remarks, Applicant argues, citing Berkheimer v. HP, Inc., “the features of the amended claims of the present application are not well-understood, routine, and conventional.” Examiner respectfully disagrees and asserts that the claimed “obtaining data of a task sound that accompanies the task and that has been collected” steps/functions of the independent claims would not account for significantly more than the abstract idea because receiving data and displaying/presenting data (See MPEP 2106.05) have been identified as well-known, routine, and conventional steps/functions to one of ordinary skill in the art. Citing Option 2 of the argued Berkheimer analysis “Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information)” has been recognized by the courts as a well-understood, routine, and conventional computer function. Applicant’s arguments are not persuasive.
Finally, citing the above discussed Ex parte Desjardins (Appeal 2024-000567), Applicant argues on pages 27-28 of the provided remarks, that “the amended claim 1 thus includes additional elements that amount to significantly more than any alleged judicial exception itself, especially in light of the newly-implemented claim amendments in this paper.” Examiner respectfully disagrees and begins by asserting that the underlined portion of cited Ex parte Desjardins is not applicable to the Step 2B argument as Applicant has referenced 35 USC 102, 35 USC 103, and 35 USC 112 arguments. Further, as stated above, the additional elements cited as “obtaining data of a task sound that accompanies the task and that has been collected” steps/functions of the independent claims would not account for significantly more than the abstract idea because receiving data and displaying/presenting data (See MPEP 2106.05) have been identified as well-known, routine, and conventional steps/functions to one of ordinary skill in the art. The 35 USC 101 rejection is maintained. Applicant’s arguments are not persuasive.
Applicant's arguments regarding claim rejections under 35 USC 103 filed 04/22/26 have been fully considered but they are not persuasive.
On pages 28-32 of the provided remarks, Applicant argues that the cited prior art does not disclose the amended claim limitations. Beginning on page 29 of the provided remarks, Applicant argues that the cited references “do not disclose identifying optical properties, e.g., transmission, based on acoustic properties”. Examiner respectfully disagrees and begins by asserting that the amended claims do not recite the argued identifying of optical properties, e.g., transmission. Additionally, the cited Zhang discloses the claimed estimation based on the feature determination of whether the worker is performing a task by inputting the data of the task sound into a first model that has been trained. Applicant’s arguments are not persuasive.
Continuing on pages 29-30 of the provided remarks, Applicant argues that cited McEldowney does not disclose “estimating whether a worker is performing a task in which a transparent object is handled. Zhang also does not disclose estimating whether a worker is performing a task in which a transparent object is handled.” Examiner respectfully disagrees and asserts that the present rejection against the claims asserts that Zhang discloses “estimating whether a worker is performing a task in which a object is handled”, therefore Applicant’s argument against “transparent objects” is moot. Further McEldowney discloses per cited paragraph 0030 the determination of the material of the object being handled. This determination of material of the object under broadest reasonable interpretation includes the transparency of the object if the object is made of a transparent material. Therefore, the cited prior art discloses the claimed limitations. Applicant’s arguments are not persuasive.
Continuing on page 30 of the provided remarks, Applicant argues “the target to be estimated in Zhang differs from the target to be estimated as recited in independent claim 1 of the present application.” Examiner respectfully disagrees and asserts that the general monitoring for detecting conditions of Zhang is applicable to the claimed estimating of a task within the independent claims. Additionally, as stated above, the combination of Zhang and McEldowney disclose the argued estimating a task in which a transparent object is handled by inputting task sound data into a trained first model. Applicant’s arguments are not persuasive.
Continuing on page 30 of the provided remarks, Applicant argues that cited McEldowney “is not directed to a technique for “estimating based on task sounds”. Examiner respectfully disagrees and asserts that per paragraph [0002] of McEldowney, the application is directed to “identifying a feature of an object”. This identification is performed utilizing audio data, per cited paragraph [0087], which is analogous to the estimation based on task sounds of the present invention. Applicant’s arguments are not persuasive.
Further, Applicant argues “a person having ordinary skill in the subject art would be led to add a polarization camera or depth sensing system to Zhang”. Examiner respectfully disagrees and asserts that the monitoring and addressing of concerns method of Zhang would be enhanced utilizing the argued polarization camera or depth sensing system including transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003). Applicant’s arguments are not persuasive.
Finally, on page 31 of the provided remarks, Applicant argues the combination of Zhang and McEldowney, stating “Examiner’s asserted combination does not cause the technical concept of McEldowney to be in consistency with the aim of the present application, and is believed to be applied based on hindsight after reviewing the claims of the present application.” In response to applicant's argument that the examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). Examiner maintains the 35 USC 103 rejection. Applicant’s arguments are not persuasive.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter;
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter. If the claim does fall within one of the statutory categories, it must then be determined whether the claim is directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea), and if so, it must additionally be determined whether the claim is a patent-eligible application of the exception. If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim amounts to significantly more than the abstract idea itself.
Step 1: Independent claims 1 (method), 9 (system), and dependent claims 2-8 and 10, respectively, fall within at least one of the four statutory categories of 35 U.S.C. 101: (i) process; (ii) machine; (iii) manufacture; or (iv) composition of matter. Claim 1 is directed to a method (i.e. process) and claim 9 is directed to a system (i.e. machine).
Step 2A Prong 1: The independent claims recite estimation method, performed by a computer, of estimating a task performed by a worker, the estimation method comprising: obtaining data of a task sound that accompanies the task and that has been collected; extracting a feature from the data of the task sound by converting the task sound into the feature for model processing; and estimating, based on the feature, whether the worker is performing a task in which a transparent object is handled, by inputting the data of the task sound into a first model that has been trained (Certain Method of Organizing Human Activity & Mental Process), which are considered to be abstract ideas (See PEG 2019 and MPEP 2106.05). [Examiner notes the underlined limitations above recite the abstract idea].
The steps/functions disclosed above and in the independent claims recite the abstract idea of Certain Methods of Organizing Human Activity because the claimed limitations are performing an estimation method estimating, based on determined features, whether the worker is performing a task in which a transparent object is handled, which is managing personal behavior. The Applicant’s claimed limitations are estimating a task performed by a worker, which recite the abstract idea of Organizing Human Activity.
The steps/functions disclosed above and in the independent claims recite the abstract idea of Mental Process because the claimed limitations are performing an estimation method extracting a feature from the data of the task sound by converting the task sound into the feature and estimating whether the worker is performing a task in which a transparent object is handled, which is an observation, judgment, and evaluation of the human mind. The Applicant’s claimed limitations are estimating a task performed by a worker, which recite the abstract idea of Mental Process.
In addition, dependent claims 2-8 further narrow the abstract idea and recite further defining the data obtained; estimating whether the worker is performing a task based on the result of estimating using a first and second model; based on the task sound and image of the task; based on similarity between a feature of the task sound output; and determining the feature of a task sound of a task. These processes are similar to the abstract idea noted in the independent claims because they further the limitations of the independent claims which recite a certain method of organizing human activity which include commercial interactions such as managing personal behavior as well as mental processes. Accordingly, these claim elements do not serve to confer subject matter eligibility to the claims since they recite abstract ideas. Dependent claim 10 will be discussed in Prong 2 analysis below.
Step 2A Prong 2: In this application, the above “obtaining data of a task sound that accompanies the task and that has been collected” steps/functions of the independent claims would not account for additional elements that integrate the judicial exception (e.g. abstract idea) into a practical application because receiving/storing data and displaying data merely add insignificant extra-solution activity and merely adds the words to apply it with the judicial exception. Also, the claimed “a computer; An estimation device that estimates a task performed by a worker, the estimation device comprising: a processor; and a memory, wherein using the memory, the processor” would not account for additional elements that integrate the judicial exception (e.g. abstract idea) into a practical application because the claimed structure merely adds the words to apply it with the judicial exception and mere instructions to implement an abstract idea on a computer (See PEG 2019 and MPEP 2106.05).
Independent claims 1 and 9 recite the following limitation, “inputting the data of the task sound into a first model that has been trained.” The “a first model that has been trained” are recited so generically (no details whatsoever are provided other than that they are general purpose computing components) that they represent no more than mere instructions to apply the judicial exception on a computer. These limitations can also be viewed as nothing more than an attempt to generally link the use of the judicial exception to the technological environment of a computer. These limitations would not account for additional elements that integrate the judicial exception (e.g. abstract idea) into a practical application because the claimed structure merely adds the words to apply it with the judicial exception and mere instructions to implement an abstract idea on a computer (See PEG 2019 and MPEP 2106.05).
In addition, dependent claims 2-8 further narrow the abstract idea and dependent claims 2, 3, 7, and 10 additionally recite “obtaining data of an image in which the worker performing the task appears, the data of the image corresponding to the data of the task sound”; “obtaining data of an image in which the worker performing the task appears, the data of the image corresponding to the data of the task sound”; “storing, in the storage, the feature of the task sound of the task in which the non-transparent object is handled as the feature of the task sound that can be erroneously estimated” which do not account for additional elements that integrate the judicial exception (e.g. abstract idea) into a practical application because receiving/storing data and displaying data merely add insignificant extra-solution activity and the claimed “computer”, “storage”, and “A non-transitory computer-readable recording medium having recorded thereon a program for causing a computer to execute the estimation method” which do not account for additional elements that integrate the judicial exception (e.g. abstract idea) into a practical application because the claimed structure merely adds the words to apply it with the judicial exception and mere instructions to implement an abstract idea on a computer (See PEG 2019 and MPEP 2106.05).
The claimed “a computer; An estimation device that estimates a task performed by a worker, the estimation device comprising: a processor; and a memory, wherein using the memory, the processor” are recited so generically (no details whatsoever are provided other than that they are general purpose computing components and regular office supplies) that they represent no more than mere instructions to apply the judicial exception on a computer. These limitations can also be viewed as nothing more than an attempt to generally link the use of the judicial exception to the technological environment of a computer. Even when viewed in combination, the additional elements in the claims do no more than use the computer components as a tool. There is no change to the computers and other technology that is recited in the claim, and thus the claims do not improve computer functionality or other technology (See PEG 2019).
Step 2B: When analyzing the additional element(s) and/or combination of elements in the claim(s) other than the abstract idea per se the claim limitations amount(s) to no more than: a general link of the use of an abstract idea to a particular technological environment and merely amounts to the application or instructions to apply the abstract idea on a computer (See MPEP 2106.05 and PEG 2019). Further, method claims 1-8 & 10; and System claim 9 recite “a computer; An estimation device that estimates a task performed by a worker, the estimation device comprising: a processor; and a memory, wherein using the memory, the processor”; however, these elements merely facilitate the claimed functions at a high level of generality and they perform conventional functions and are considered to be general purpose computer components which is supported by Applicant’s specification in Page 16 lines 16-27; Page 19 lines 29-31; Pages 46-48; and Figures 1 & 31. The Applicant’s claimed additional elements are mere instructions to implement the abstract idea on a general purpose computer and generally link of the use of an abstract idea to a particular technological environment. Also, the above “obtaining data of a task sound that accompanies the task and that has been collected” steps/functions of the independent claims would not account for significantly more than the abstract idea because receiving data and displaying/presenting data (See MPEP 2106.05) have been identified as well-known, routine, and conventional steps/functions to one of ordinary skill in the art. When viewed as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself.
Next, when the “first model” & “second model” is evaluated as an additional element, this feature is recited at a high level of generality and encompasses well-understood, routine, and conventional prior art activity. Accordingly, the use of a first model to estimate whether a worker is performing a task does not add significantly more to the claim.
In addition, claims 2-8 further narrow the abstract idea identified in the independent claims. The Examiner notes that the dependent claims merely further define the data being analyzed and how the data is being analyzed. Similarly, claims 2, 3, 7, and 10 additionally recite “obtaining data of an image in which the worker performing the task appears, the data of the image corresponding to the data of the task sound”; “obtaining data of an image in which the worker performing the task appears, the data of the image corresponding to the data of the task sound”; “storing, in the storage, the feature of the task sound of the task in which the non-transparent object is handled as the feature of the task sound that can be erroneously estimated” which do not account for additional elements that amount to significantly more than the abstract idea because receiving data and displaying/presenting data (See MPEP 2106.05) have been identified as well-known, routine, and conventional steps/functions to one of ordinary skill in the art and the claimed “computer”, “storage”, and “A non-transitory computer-readable recording medium having recorded thereon a program for causing a computer to execute the estimation method” which do not account for additional elements that amount to significantly more than the abstract idea because the claimed structure merely amounts to the application or instructions to apply the abstract idea on a computer and does not move beyond a general link of the use of an abstract idea to a particular technological environment (See MPEP 2106.05). The additional limitations of the independent and dependent claim(s) when considered individually and as an ordered combination do not amount to significantly more than the abstract idea. The examiner has considered the dependent claims in a full analysis including the additional limitations individually and in combination as analyzed in the independent claim(s). Therefore, the claim(s) are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang (U.S 2021/0027485 A1) in view of McEdowney (U.S 2021/0084206 A1).
Claims 1, 9, and 10
Regarding Claim 1, Zhang discloses the following:
An estimation method, performed by a computer, of estimating a task performed by a worker, the estimation method comprising [see at least Paragraph 0067 for reference to the system for status monitoring using machine learning and machine vision that can be applied to monitor stores, parks, office buildings, and other locations; Figures 5-6 and related text regarding an example process for training and using machine learning models]
obtaining data of a task sound that accompanies the task and that has been collected [see at least Paragraph 0072 for reference to image and audio data being obtained from cameras installed or from existing camera and microphone systems; Paragraph 0078 for reference to the system including sensors to monitor the environment including microphones to detect ambient sound in an area, conversations of employees (e.g., clerks taking orders at the register), or other audio in the restaurant; Paragraph 0079 for reference to the computing system receiving the image data and the audio data and can use a data pre-processor to extract feature values to be provided as input; Figure 1 and related text regarding item 116 ‘audio data’]
extracting a feature from the data of the task sound by converting the task sound into the feature for model processing [see at least Paragraph 0079 for reference to the computing system receives the image data and the audio data and can use a data pre - processor to extract feature values to be provided as input; Paragraph 0080 for reference to one or more machine learning models process the input data representing the sensed parameters of the environment of the restaurant; Paragraph 0101 for reference to computer system can extract, from the list of detected objects, statistics and measures about the different objects detected and apply rules, threshold, or other evaluation techniques to determine whether certain issues are present in the restaurant]
estimating, based on the feature, whether the worker is performing a task in which a object is handled, by inputting the data of the task sound into a first model that has been trained [see at least Paragraph 0031 for reference to the public area being a checkout area; Paragraph 0034 for reference to the output identifying the location of the detected condition within the public area; Paragraph 0080 for reference to a model being used to process input from the audio data to detect whether noise levels are above a threshold level or whether a worker at a store performed an action for assisting a customer as well as detecting different types of objects; Paragraph 0081 for reference to model being trained to perform multiple tasks such as recognize the type of object present and determine the status of an object; Paragraph 0110 for reference to the system using audio data to detect events or conditions at monitored locations; Paragraph 0110 for reference to the computer system localizing the condition detected from audio data using the results from the neural network models]
While Zhang discloses the limitations above, it does not recite estimating whether the worker is performing a task in which a transparent object is handled.
However, McEldowney discloses the following:
estimating whether the worker is performing a task in which a transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Regarding claims 9 and 10, the claims recite limitations already addressed by the rejection of claim 1. Regarding claim 9, Zhang teaches an estimation device comprising a processor; and a memory [Paragraph 0067 & Figure 1]. Regarding claim 10, Zhang teaches a non-transitory computer-readable recording medium having recorded thereon a program for causing a computer to execute the estimation method [Paragraph 0165]. Therefore, claims 9 and 10 are rejected as being unpatentable in view of Zhang and McEldowney.
Claim 2
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 2, Zhang discloses the following:
obtaining data of an image in which the worker performing the task appears, the data of the image corresponding to the data of the task sound [see at least Paragraph 0077 for reference to camera providing image data representing the images to the computer system; Paragraph 0079 for reference to the computing system receiving image data and using the data pre-processor to extract feature values to be provided as input; Figure 1 and related text regarding item 114a/b ‘image data’; Figure 5 and related text regarding item 502 ‘OBTAIN IMAGE DATA’; Figure 6 and related text regarding item 602 ‘OBTAIN IMAGE DATA’]
estimating whether the worker is performing the task in which the object is handled, by inputting the data of the image into a second model that has been trained [see at least Paragraph 0080 for reference to one model being configured to detect people, chairs, tables, food, and litter can be used to process image data for the display case; Paragraph 0110 for reference to the computer system localizing the condition detected from audio data using the results from the neural network models in processing image data; Paragraph 0133 for reference to the training process can cause a model to learn, based on the examples of the image data, to detect different conditions; Paragraph 0153 for reference the image data is processed using one or more machine learning models; Figure 5 and related text regarding item 508 ‘TRAIN MACHINE LEARNING MODEL(S)’; Figure 6 and related text regarding item 604 ‘PROCESS IMAGE DATA USING MACHINE LEARNING MODELS’]
estimating whether the worker is performing the task in which the object is handled, based on a result of the estimating using the first model and a result of the estimating using the second model [see at least Paragraph 0110 for reference to the system using audio data as well as image data to detect events and conditions at monitored locations; Paragraph 0110 for reference to the computer system localizing the condition detected from audio data using the results from the neural network models in processing image data]
While Zhang discloses the limitations above, it does not recite estimating whether the worker is performing a task in which a transparent object is handled.
However, McEldowney discloses the following:
obtaining data of an image [see at least Paragraph 0032 for reference to the polarization capture device being configured to capture images of an object; Figure 5 and related text regarding item 510 ‘Capturing image data’]
estimating whether the worker is performing a task in which a transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Claim 3
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 3, Zhang discloses the following:
obtaining data of an image in which the worker performing the task appears, the data of the image corresponding to the data of the task sound [see at least Paragraph 0077 for reference to camera providing image data representing the images to the computer system; Paragraph 0079 for reference to the computing system receiving image data and using the data pre-processor to extract feature values to be provided as input; Figure 1 and related text regarding item 114a/b ‘image data’; Figure 5 and related text regarding item 502 ‘OBTAIN IMAGE DATA’; Figure 6 and related text regarding item 602 ‘OBTAIN IMAGE DATA’]
estimating whether the worker is performing the task in which the object is handled, by inputting the data of the task sound and the data of the image into the first model [see at least Paragraph 0110 for reference to the system using audio data as well as image data to detect events and conditions at monitored locations; Paragraph 0110 for reference to the computer system localizing the condition detected from audio data using the results from the neural network models in processing image data]
While Zhang discloses the limitations above, it does not recite estimating whether the worker is performing a task in which a transparent object is handled.
However, McEldowney discloses the following:
obtaining data of an image [see at least Paragraph 0032 for reference to the polarization capture device being configured to capture images of an object; Figure 5 and related text regarding item 510 ‘Capturing image data’]
estimating whether the worker is performing a task in which a transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Claim 4
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 4, Zhang discloses the following:
estimating whether the worker is performing the task in which the object is handled, based on a similarity between a feature of the task sound output from the first model and a feature, stored in storage in advance, of a task sound of the task in which the object is handled [see at least Paragraph 0079 for reference to the computing system receiving the image data and the audio data and can use a data pre-processor to extract feature values to be provided as input; Paragraph 0082 for reference to the models processing image data using the faster R-CNN object detection and recognition framework; Paragraph 0086 for reference to feature maps generated through convolution being provided to both the object detection classifier and the region proposal network; Figure 6 and related text regarding item 608 ‘OBTAIN CLASSIFICATIONS, CONFIDENCE SCORES’]
While Zhang discloses the limitations above, it does not recite estimating whether the worker is performing a task in which a transparent object is handled, based on a similarity between a feature of the task sound output from the first model and a feature, stored in storage in advance, of a task sound of the task in which the transparent object is handled.
However, McEldowney discloses the following:
estimating whether the worker is performing a task in which a transparent object is handled based on a similarity between a feature of the task sound output from the first model and a feature, stored in storage in advance, of a task sound of the task in which the transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0058 for reference to the feature extractor may refer to a table storing various properties of various materials including wood to identify the acoustic properties of the wood material; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Claim 5
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 5, Zhang discloses the following:
estimating whether the worker is performing the task in which the object is handled, based on a similarity of a feature of the task sound output from the first model to each of (i) a feature of a task sound, stored in advance in storage, of the task in which the object is handled and (ii) a feature of a task sound, stored in advance in the storage, from which the worker can be erroneously estimated to be performing the task in which the object is handled [see at least Paragraph 0079 for reference to the computing system receiving the image data and the audio data and can use a data pre-processor to extract feature values to be provided as input; Paragraph 0082 for reference to the models processing image data using the faster R-CNN object detection and recognition framework; Paragraph 0086 for reference to feature maps generated through convolution being provided to both the object detection classifier and the region proposal network; Paragraph 0101 for reference to the computer system can extract, from the list of detected objects, statistics and measures about the different objects detected and apply rules, threshold, or other evaluation techniques to determine whether certain issues are present in the restaurant; Figure 6 and related text regarding item 608 ‘OBTAIN CLASSIFICATIONS, CONFIDENCE SCORES’]
While Zhang discloses the limitations above, it does not recite estimating whether the worker is performing a task in which a transparent object is handled, based on a similarity of a feature of the task sound output from the first model to each of (i) a feature of a task sound, stored in advance in storage, of the task in which the transparent object is handled and (ii) a feature of a task sound, stored in advance in the storage, from which the worker can be erroneously estimated to be performing the task in which the transparent object is handled.
However, McEldowney discloses the following:
estimating whether the worker is performing a task in which a transparent object is handled, based on a similarity of a feature of the task sound output from the first model to each of (i) a feature of a task sound, stored in advance in storage, of the task in which the transparent object is handled and (ii) a feature of a task sound, stored in advance in the storage, from which the worker can be erroneously estimated to be performing the task in which the transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0058 for reference to the feature extractor may refer to a table storing various properties of various materials including wood to identify the acoustic properties of the wood material; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Claim 6
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 6, Zhang discloses the following:
wherein the worker is estimated to be performing the task in which the object is handled when the similarity of the feature of the task sound output from the first model to the feature of the task sound of the task in which the object is handled exceeds the similarity to the feature of the task sound from which the worker can be erroneously estimated to be performing the task in which the object is handled [see at least Paragraph 0101 for reference to the computer system can extract, from the list of detected objects, statistics and measures about the different objects detected and apply rules, threshold, or other evaluation techniques to determine whether certain issues are present in the restaurant; Paragraph 0136 for reference to the models can be trained to distinguish types of variations in images that are within the normal or expected range of conditions (e.g., image data showing different arrangements of people and food around occupied tables) from items in images that show changes that need corrective action; Paragraph 0139 for reference to the computer system generating threshold parameters for the confidence level for a certain condition before action is requested; Figure 5 and related text regarding item 518 ‘GENERATE THRESHOLD, RULES, POST-PROCESSING PARAMETERS’]
While Zhang discloses the limitations above, it does not recite wherein the worker is estimated to be performing the task in which the transparent object is handled when the similarity of the feature of the task sound output from the first model to the feature of the task sound of the task in which the transparent object is handled exceeds the similarity to the feature of the task sound from which the worker can be erroneously estimated to be performing the task in which the transparent object is handled.
However, McEldowney discloses the following:
wherein the worker is estimated to be performing the task in which the transparent object is handled when the similarity of the feature of the task sound output from the first model to the feature of the task sound of the task in which the transparent object is handled exceeds the similarity to the feature of the task sound from which the worker can be erroneously estimated to be performing the task in which the transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0058 for reference to the feature extractor may refer to a table storing various properties of various materials including wood to identify the acoustic properties of the wood material; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Claim 7
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 7, Zhang discloses the following:
when a similarity of (i) a feature of a task sound of a task in which a non-transparent object different from the object is handled, the feature being obtained by inputting, to the first model, data of the task sound of the task in which the non-transparent object is handled, to (ii) the feature of the task sound of the task in which the object is handled, exceeds a threshold, determining that the task sound of the task in which the non-transparent object is handled is a task sound that can be erroneously estimated as the task sound of the task in which the object is handled [see at least Paragraph 0079 for reference to the computing system receiving the image data and the audio data and can use a data pre-processor to extract feature values to be provided as input; Paragraph 0082 for reference to the models processing image data using the faster R-CNN object detection and recognition framework; Paragraph 0086 for reference to feature maps generated through convolution being provided to both the object detection classifier and the region proposal network; Paragraph 0101 for reference to the computer system can extract, from the list of detected objects, statistics and measures about the different objects detected and apply rules, threshold, or other evaluation techniques to determine whether certain issues are present in the restaurant; Figure 6 and related text regarding item 608 ‘OBTAIN CLASSIFICATIONS, CONFIDENCE SCORES’]
storing, in the storage, the feature of the task sound of the task in which the non-transparent object is handled as the feature of the task sound that can be erroneously estimated [see at least Paragraph 0106 for reference to the computer system storing the information specifying the pending and completed tasks in a task data store and this can be used to provide a variety of analytics data for the restaurant; Figure 1 and related text regarding item 129 ‘task data store’]
While Zhang discloses the limitations above, it does not recite when a similarity of (i) a feature of a task sound of a task in which a non-transparent object different from the transparent object is handled, the feature being obtained by inputting, to the first model, data of the task sound of the task in which the non-transparent object is handled, to (ii) the feature of the task sound of the task in which the transparent object is handled, exceeds a threshold, determining that the task sound of the task in which the non-transparent object is handled is a task sound that can be erroneously estimated as the task sound of the task in which the transparent object is handled.
However, McEldowney discloses the following:
when a similarity of (i) a feature of a task sound of a task in which a non-transparent object different from the transparent object is handled, the feature being obtained by inputting, to the first model, data of the task sound of the task in which the non-transparent object is handled, to (ii) the feature of the task sound of the task in which the transparent object is handled, exceeds a threshold, determining that the task sound of the task in which the non-transparent object is handled is a task sound that can be erroneously estimated as the task sound of the task in which the transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0058 for reference to the feature extractor may refer to a table storing various properties of various materials including wood to identify the acoustic properties of the wood material; Paragraph 0060 for reference to any combination of such information being input into a machine learning system (e.g., a Convolutional Neural Network (“CNN”)) implemented in the feature extractor to extract features or characteristics of the object such as the shape, texture, surface roughness, material, etc.; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Claim 8
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 8, Zhang discloses the following:
wherein the data of the task sound includes data of a sound in an inaudible range [see at least Paragraph 0072 for reference to image and audio data being obtained from cameras installed or from existing camera and microphone systems; Paragraph 0078 for reference to the system including sensors to monitor the environment including microphones to detect ambient sound in an area, conversations of employees (e.g., clerks taking orders at the register), or other audio in the restaurant; Paragraph 0138 for reference to audio data being processed to detect unusual or undesirable conditions by detecting when music is played to loudly or too softly, when speech or environmental noise is too high, and so on; Figure 1 and related text regarding item 116 ‘audio data’]
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Weerasinghe, Ittepana Payagalage Tharindu Rasanga. "Automated construction worker performance and tool-time measuring model using RGB depth camera and audio microphone array system." (2013).
DOCUMENT ID
INVENTOR(S)
TITLE
CN 112202655 A
Ma et al.
Intelligent Electric Appliance, Image Identification Method, Electronic Device And Storage Medium
US 20210356572 A1
Kadambi et al.
SYSTEMS AND METHODS FOR AUGMENTATION OF SENSOR SYSTEMS AND IMAGING SYSTEMS WITH POLARIZATION
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KRISTIN ELIZABETH GAVIN whose telephone number is (571)270-7019. The examiner can normally be reached M-F 7:30-4:30 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jerry O'Connor can be reached at 571-272-6787. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KRISTIN E GAVIN/Primary Examiner, Art Unit 3624
Prosecution Insights