DETAILED ACTION
This non-final Office action is responsive to the application filed December 13th, 2024. Claims 1-10 are presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/13/24 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“an obtainer that obtains data” in claim 9; and
“an estimator that estimates” in claim 9.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
When looking to the specification, the hardware structure associated with the “obtainer” and “estimator” is being interpreted as “processor,” please see at least Fig. 1 and Page 17 lines 8-12, 14-24, & 26-35 of the instant specification. The corresponding algorithm of the “obtainer” and “estimator” can be found in at least Fig. 1 and Page 17 lines 8-12, 14-24, & 26-35. This is to be the structure and algorithm required for the claim, or equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter;
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter. If the claim does fall within one of the statutory categories, it must then be determined whether the claim is directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea), and if so, it must additionally be determined whether the claim is a patent-eligible application of the exception. If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim amounts to significantly more than the abstract idea itself.
Step 1: Independent claims 1 (method), 9 (system), and dependent claims 2-8 and 10, respectively, fall within at least one of the four statutory categories of 35 U.S.C. 101: (i) process; (ii) machine; (iii) manufacture; or (iv) composition of matter. Claim 1 is directed to a method (i.e. process) and claim 9 is directed to a system (i.e. machine).
Step 2A Prong 1: The independent claims recite estimation method, performed by a computer, of estimating a task performed by a worker, the estimation method comprising: obtaining data of a task sound that accompanies the task and that has been collected; and estimating whether the worker is performing a task in which a transparent object is handled, by inputting the data of the task sound into a first model that has been trained (Certain Method of Organizing Human Activity & Mental Process), which are considered to be abstract ideas (See PEG 2019 and MPEP 2106.05). [Examiner notes the underlined limitations above recite the abstract idea].
The steps/functions disclosed above and in the independent claims recite the abstract idea of Certain Methods of Organizing Human Activity because the claimed limitations are performing an estimation method estimating whether the worker is performing a task in which a transparent object is handled, which is managing personal behavior. The Applicant’s claimed limitations are estimating a task performed by a worker, which recite the abstract idea of Organizing Human Activity.
The steps/functions disclosed above and in the independent claims recite the abstract idea of Mental Process because the claimed limitations are performing an estimation method estimating whether the worker is performing a task in which a transparent object is handled, which is an observation, judgment, and evaluation of the human mind. The Applicant’s claimed limitations are estimating a task performed by a worker, which recite the abstract idea of Mental Process.
In addition, dependent claims 2-8 further narrow the abstract idea and recite further defining the data obtained; estimating whether the worker is performing a task based on the result of estimating using a first and second model; based on the task sound and image of the task; based on similarity between a feature of the task sound output; and determining the feature of a task sound of a task. These processes are similar to the abstract idea noted in the independent claims because they further the limitations of the independent claims which recite a certain method of organizing human activity which include commercial interactions such as managing personal behavior as well as mental processes. Accordingly, these claim elements do not serve to confer subject matter eligibility to the claims since they recite abstract ideas. Dependent claim 10 will be discussed in Prong 2 analysis below.
Step 2A Prong 2: In this application, the above “obtaining data of a task sound that accompanies the task and that has been collected” steps/functions of the independent claims would not account for additional elements that integrate the judicial exception (e.g. abstract idea) into a practical application because receiving/storing data and displaying data merely add insignificant extra-solution activity and merely adds the words to apply it with the judicial exception. Also, the claimed “a computer; An estimation device that estimates a task performed by a worker, the estimation device comprising: an obtainer; an estimator” would not account for additional elements that integrate the judicial exception (e.g. abstract idea) into a practical application because the claimed structure merely adds the words to apply it with the judicial exception and mere instructions to implement an abstract idea on a computer (See PEG 2019 and MPEP 2106.05).
Independent claims 1 and 9 recite the following limitation, “inputting the data of the task sound into a first model that has been trained.” The “a first model that has been trained” are recited so generically (no details whatsoever are provided other than that they are general purpose computing components) that they represent no more than mere instructions to apply the judicial exception on a computer. These limitations can also be viewed as nothing more than an attempt to generally link the use of the judicial exception to the technological environment of a computer. These limitations would not account for additional elements that integrate the judicial exception (e.g. abstract idea) into a practical application because the claimed structure merely adds the words to apply it with the judicial exception and mere instructions to implement an abstract idea on a computer (See PEG 2019 and MPEP 2106.05).
In addition, dependent claims 2-8 further narrow the abstract idea and dependent claims 2, 3, 7, and 10 additionally recite “obtaining data of an image in which the worker performing the task appears, the data of the image corresponding to the data of the task sound”; “obtaining data of an image in which the worker performing the task appears, the data of the image corresponding to the data of the task sound”; “storing, in the storage, the feature of the task sound of the task in which the non-transparent object is handled as the feature of the task sound that can be erroneously estimated” which do not account for additional elements that integrate the judicial exception (e.g. abstract idea) into a practical application because receiving/storing data and displaying data merely add insignificant extra-solution activity and the claimed “computer”, “storage”, and “A non-transitory computer-readable recording medium having recorded thereon a program for causing a computer to execute the estimation method” which do not account for additional elements that integrate the judicial exception (e.g. abstract idea) into a practical application because the claimed structure merely adds the words to apply it with the judicial exception and mere instructions to implement an abstract idea on a computer (See PEG 2019 and MPEP 2106.05).
The claimed “a computer; An estimation device that estimates a task performed by a worker, the estimation device comprising: an obtainer; an estimator” are recited so generically (no details whatsoever are provided other than that they are general purpose computing components and regular office supplies) that they represent no more than mere instructions to apply the judicial exception on a computer. These limitations can also be viewed as nothing more than an attempt to generally link the use of the judicial exception to the technological environment of a computer. Even when viewed in combination, the additional elements in the claims do no more than use the computer components as a tool. There is no change to the computers and other technology that is recited in the claim, and thus the claims do not improve computer functionality or other technology (See PEG 2019).
Step 2B: When analyzing the additional element(s) and/or combination of elements in the claim(s) other than the abstract idea per se the claim limitations amount(s) to no more than: a general link of the use of an abstract idea to a particular technological environment and merely amounts to the application or instructions to apply the abstract idea on a computer (See MPEP 2106.05 and PEG 2019). Further, method claims 1-8 & 10; and System claim 9 recite “a computer; An estimation device that estimates a task performed by a worker, the estimation device comprising: an obtainer; an estimator”; however, these elements merely facilitate the claimed functions at a high level of generality and they perform conventional functions and are considered to be general purpose computer components which is supported by Applicant’s specification in Page 16 lines 16-27; Page 19 lines 29-31; Pages 46-48; and Figures 1 & 31. The Applicant’s claimed additional elements are mere instructions to implement the abstract idea on a general purpose computer and generally link of the use of an abstract idea to a particular technological environment. Also, the above “obtaining data of a task sound that accompanies the task and that has been collected” steps/functions of the independent claims would not account for significantly more than the abstract idea because receiving data and displaying/presenting data (See MPEP 2106.05) have been identified as well-known, routine, and conventional steps/functions to one of ordinary skill in the art. When viewed as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself.
Next, when the “first model” & “second model” is evaluated as an additional element, this feature is recited at a high level of generality and encompasses well-understood, routine, and conventional prior art activity. Accordingly, the use of a first model to estimate whether a worker is performing a task does not add significantly more to the claim.
In addition, claims 2-8 further narrow the abstract idea identified in the independent claims. The Examiner notes that the dependent claims merely further define the data being analyzed and how the data is being analyzed. Similarly, claims 2, 3, 7, and 10 additionally recite “obtaining data of an image in which the worker performing the task appears, the data of the image corresponding to the data of the task sound”; “obtaining data of an image in which the worker performing the task appears, the data of the image corresponding to the data of the task sound”; “storing, in the storage, the feature of the task sound of the task in which the non-transparent object is handled as the feature of the task sound that can be erroneously estimated” which do not account for additional elements that amount to significantly more than the abstract idea because receiving data and displaying/presenting data (See MPEP 2106.05) have been identified as well-known, routine, and conventional steps/functions to one of ordinary skill in the art and the claimed “computer”, “storage”, and “A non-transitory computer-readable recording medium having recorded thereon a program for causing a computer to execute the estimation method” which do not account for additional elements that amount to significantly more than the abstract idea because the claimed structure merely amounts to the application or instructions to apply the abstract idea on a computer and does not move beyond a general link of the use of an abstract idea to a particular technological environment (See MPEP 2106.05). The additional limitations of the independent and dependent claim(s) when considered individually and as an ordered combination do not amount to significantly more than the abstract idea. The examiner has considered the dependent claims in a full analysis including the additional limitations individually and in combination as analyzed in the independent claim(s). Therefore, the claim(s) are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang (U.S 2021/0027485 A1) in view of McEdowney (U.S 2021/0084206 A1).
Claims 1, 9, and 10
Regarding Claim 1, Zhang discloses the following:
An estimation method, performed by a computer, of estimating a task performed by a worker, the estimation method comprising [see at least Paragraph 0067 for reference to the system for status monitoring using machine learning and machine vision that can be applied to monitor stores, parks, office buildings, and other locations; Figures 5-6 and related text regarding an example process for training and using machine learning models]
obtaining data of a task sound that accompanies the task and that has been collected [see at least Paragraph 0072 for reference to image and audio data being obtained from cameras installed or from existing camera and microphone systems; Paragraph 0078 for reference to the system including sensors to monitor the environment including microphones to detect ambient sound in an area, conversations of employees (e.g., clerks taking orders at the register), or other audio in the restaurant; Paragraph 0079 for reference to the computing system receiving the image data and the audio data and can use a data pre-processor to extract feature values to be provided as input; Figure 1 and related text regarding item 116 ‘audio data’]
estimating whether the worker is performing a task in which a object is handled, by inputting the data of the task sound into a first model that has been trained [see at least Paragraph 0031 for reference to the public area being a checkout area; Paragraph 0034 for reference to the output identifying the location of the detected condition within the public area; Paragraph 0080 for reference to a model being used to process input from the audio data to detect whether noise levels are above a threshold level or whether a worker at a store performed an action for assisting a customer as well as detecting different types of objects; Paragraph 0081 for reference to model being trained to perform multiple tasks such as recognize the type of object present and determine the status of an object; Paragraph 0110 for reference to the system using audio data to detect events or conditions at monitored locations; Paragraph 0110 for reference to the computer system localizing the condition detected from audio data using the results from the neural network models]
While Zhang discloses the limitations above, it does not recite estimating whether the worker is performing a task in which a transparent object is handled.
However, McEldowney discloses the following:
estimating whether the worker is performing a task in which a transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Regarding claims 9 and 10, the claims recite limitations already addressed by the rejection of claim 1. Regarding claim 9, Zhang teaches an estimation device comprising an obtainer and estimator [Paragraph 0067 & Figure 1]. Regarding claim 10, Zhang teaches a non-transitory computer-readable recording medium having recorded thereon a program for causing a computer to execute the estimation method [Paragraph 0165]. Therefore, claims 9 and 10 are rejected as being unpatentable in view of Zhang and McEldowney.
Claim 2
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 2, Zhang discloses the following:
obtaining data of an image in which the worker performing the task appears, the data of the image corresponding to the data of the task sound [see at least Paragraph 0077 for reference to camera providing image data representing the images to the computer system; Paragraph 0079 for reference to the computing system receiving image data and using the data pre-processor to extract feature values to be provided as input; Figure 1 and related text regarding item 114a/b ‘image data’; Figure 5 and related text regarding item 502 ‘OBTAIN IMAGE DATA’; Figure 6 and related text regarding item 602 ‘OBTAIN IMAGE DATA’]
estimating whether the worker is performing the task in which the object is handled, by inputting the data of the image into a second model that has been trained [see at least Paragraph 0080 for reference to one model being configured to detect people, chairs, tables, food, and litter can be used to process image data for the display case; Paragraph 0110 for reference to the computer system localizing the condition detected from audio data using the results from the neural network models in processing image data; Paragraph 0133 for reference to the training process can cause a model to learn, based on the examples of the image data, to detect different conditions; Paragraph 0153 for reference the image data is processed using one or more machine learning models; Figure 5 and related text regarding item 508 ‘TRAIN MACHINE LEARNING MODEL(S)’; Figure 6 and related text regarding item 604 ‘PROCESS IMAGE DATA USING MACHINE LEARNING MODELS’]
estimating whether the worker is performing the task in which the object is handled, based on a result of the estimating using the first model and a result of the estimating using the second model [see at least Paragraph 0110 for reference to the system using audio data as well as image data to detect events and conditions at monitored locations; Paragraph 0110 for reference to the computer system localizing the condition detected from audio data using the results from the neural network models in processing image data]
While Zhang discloses the limitations above, it does not recite estimating whether the worker is performing a task in which a transparent object is handled.
However, McEldowney discloses the following:
obtaining data of an image [see at least Paragraph 0032 for reference to the polarization capture device being configured to capture images of an object; Figure 5 and related text regarding item 510 ‘Capturing image data’]
estimating whether the worker is performing a task in which a transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Claim 3
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 3, Zhang discloses the following:
obtaining data of an image in which the worker performing the task appears, the data of the image corresponding to the data of the task sound [see at least Paragraph 0077 for reference to camera providing image data representing the images to the computer system; Paragraph 0079 for reference to the computing system receiving image data and using the data pre-processor to extract feature values to be provided as input; Figure 1 and related text regarding item 114a/b ‘image data’; Figure 5 and related text regarding item 502 ‘OBTAIN IMAGE DATA’; Figure 6 and related text regarding item 602 ‘OBTAIN IMAGE DATA’]
estimating whether the worker is performing the task in which the object is handled, by inputting the data of the task sound and the data of the image into the first model [see at least Paragraph 0110 for reference to the system using audio data as well as image data to detect events and conditions at monitored locations; Paragraph 0110 for reference to the computer system localizing the condition detected from audio data using the results from the neural network models in processing image data]
While Zhang discloses the limitations above, it does not recite estimating whether the worker is performing a task in which a transparent object is handled.
However, McEldowney discloses the following:
obtaining data of an image [see at least Paragraph 0032 for reference to the polarization capture device being configured to capture images of an object; Figure 5 and related text regarding item 510 ‘Capturing image data’]
estimating whether the worker is performing a task in which a transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Claim 4
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 4, Zhang discloses the following:
estimating whether the worker is performing the task in which the object is handled, based on a similarity between a feature of the task sound output from the first model and a feature, stored in storage in advance, of a task sound of the task in which the object is handled [see at least Paragraph 0079 for reference to the computing system receiving the image data and the audio data and can use a data pre-processor to extract feature values to be provided as input; Paragraph 0082 for reference to the models processing image data using the faster R-CNN object detection and recognition framework; Paragraph 0086 for reference to feature maps generated through convolution being provided to both the object detection classifier and the region proposal network; Figure 6 and related text regarding item 608 ‘OBTAIN CLASSIFICATIONS, CONFIDENCE SCORES’]
While Zhang discloses the limitations above, it does not recite estimating whether the worker is performing a task in which a transparent object is handled, based on a similarity between a feature of the task sound output from the first model and a feature, stored in storage in advance, of a task sound of the task in which the transparent object is handled.
However, McEldowney discloses the following:
estimating whether the worker is performing a task in which a transparent object is handled based on a similarity between a feature of the task sound output from the first model and a feature, stored in storage in advance, of a task sound of the task in which the transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0058 for reference to the feature extractor may refer to a table storing various properties of various materials including wood to identify the acoustic properties of the wood material; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Claim 5
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 5, Zhang discloses the following:
estimating whether the worker is performing the task in which the object is handled, based on a similarity of a feature of the task sound output from the first model to each of (i) a feature of a task sound, stored in advance in storage, of the task in which the object is handled and (ii) a feature of a task sound, stored in advance in the storage, from which the worker can be erroneously estimated to be performing the task in which the object is handled [see at least Paragraph 0079 for reference to the computing system receiving the image data and the audio data and can use a data pre-processor to extract feature values to be provided as input; Paragraph 0082 for reference to the models processing image data using the faster R-CNN object detection and recognition framework; Paragraph 0086 for reference to feature maps generated through convolution being provided to both the object detection classifier and the region proposal network; Paragraph 0101 for reference to the computer system can extract, from the list of detected objects, statistics and measures about the different objects detected and apply rules, threshold, or other evaluation techniques to determine whether certain issues are present in the restaurant; Figure 6 and related text regarding item 608 ‘OBTAIN CLASSIFICATIONS, CONFIDENCE SCORES’]
While Zhang discloses the limitations above, it does not recite estimating whether the worker is performing a task in which a transparent object is handled, based on a similarity of a feature of the task sound output from the first model to each of (i) a feature of a task sound, stored in advance in storage, of the task in which the transparent object is handled and (ii) a feature of a task sound, stored in advance in the storage, from which the worker can be erroneously estimated to be performing the task in which the transparent object is handled.
However, McEldowney discloses the following:
estimating whether the worker is performing a task in which a transparent object is handled, based on a similarity of a feature of the task sound output from the first model to each of (i) a feature of a task sound, stored in advance in storage, of the task in which the transparent object is handled and (ii) a feature of a task sound, stored in advance in the storage, from which the worker can be erroneously estimated to be performing the task in which the transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0058 for reference to the feature extractor may refer to a table storing various properties of various materials including wood to identify the acoustic properties of the wood material; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Claim 6
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 6, Zhang discloses the following:
wherein the worker is estimated to be performing the task in which the object is handled when the similarity of the feature of the task sound output from the first model to the feature of the task sound of the task in which the object is handled exceeds the similarity to the feature of the task sound from which the worker can be erroneously estimated to be performing the task in which the object is handled [see at least Paragraph 0101 for reference to the computer system can extract, from the list of detected objects, statistics and measures about the different objects detected and apply rules, threshold, or other evaluation techniques to determine whether certain issues are present in the restaurant; Paragraph 0136 for reference to the models can be trained to distinguish types of variations in images that are within the normal or expected range of conditions (e.g., image data showing different arrangements of people and food around occupied tables) from items in images that show changes that need corrective action; Paragraph 0139 for reference to the computer system generating threshold parameters for the confidence level for a certain condition before action is requested; Figure 5 and related text regarding item 518 ‘GENERATE THRESHOLD, RULES, POST-PROCESSING PARAMETERS’]
While Zhang discloses the limitations above, it does not recite wherein the worker is estimated to be performing the task in which the transparent object is handled when the similarity of the feature of the task sound output from the first model to the feature of the task sound of the task in which the transparent object is handled exceeds the similarity to the feature of the task sound from which the worker can be erroneously estimated to be performing the task in which the transparent object is handled.
However, McEldowney discloses the following:
wherein the worker is estimated to be performing the task in which the transparent object is handled when the similarity of the feature of the task sound output from the first model to the feature of the task sound of the task in which the transparent object is handled exceeds the similarity to the feature of the task sound from which the worker can be erroneously estimated to be performing the task in which the transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0058 for reference to the feature extractor may refer to a table storing various properties of various materials including wood to identify the acoustic properties of the wood material; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Claim 7
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 7, Zhang discloses the following:
when a similarity of (i) a feature of a task sound of a task in which a non-transparent object different from the object is handled, the feature being obtained by inputting, to the first model, data of the task sound of the task in which the non-transparent object is handled, to (ii) the feature of the task sound of the task in which the object is handled, exceeds a threshold, determining that the task sound of the task in which the non-transparent object is handled is a task sound that can be erroneously estimated as the task sound of the task in which the object is handled [see at least Paragraph 0079 for reference to the computing system receiving the image data and the audio data and can use a data pre-processor to extract feature values to be provided as input; Paragraph 0082 for reference to the models processing image data using the faster R-CNN object detection and recognition framework; Paragraph 0086 for reference to feature maps generated through convolution being provided to both the object detection classifier and the region proposal network; Paragraph 0101 for reference to the computer system can extract, from the list of detected objects, statistics and measures about the different objects detected and apply rules, threshold, or other evaluation techniques to determine whether certain issues are present in the restaurant; Figure 6 and related text regarding item 608 ‘OBTAIN CLASSIFICATIONS, CONFIDENCE SCORES’]
storing, in the storage, the feature of the task sound of the task in which the non-transparent object is handled as the feature of the task sound that can be erroneously estimated [see at least Paragraph 0106 for reference to the computer system storing the information specifying the pending and completed tasks in a task data store and this can be used to provide a variety of analytics data for the restaurant; Figure 1 and related text regarding item 129 ‘task data store’]
While Zhang discloses the limitations above, it does not recite when a similarity of (i) a feature of a task sound of a task in which a non-transparent object different from the transparent object is handled, the feature being obtained by inputting, to the first model, data of the task sound of the task in which the non-transparent object is handled, to (ii) the feature of the task sound of the task in which the transparent object is handled, exceeds a threshold, determining that the task sound of the task in which the non-transparent object is handled is a task sound that can be erroneously estimated as the task sound of the task in which the transparent object is handled.
However, McEldowney discloses the following:
when a similarity of (i) a feature of a task sound of a task in which a non-transparent object different from the transparent object is handled, the feature being obtained by inputting, to the first model, data of the task sound of the task in which the non-transparent object is handled, to (ii) the feature of the task sound of the task in which the transparent object is handled, exceeds a threshold, determining that the task sound of the task in which the non-transparent object is handled is a task sound that can be erroneously estimated as the task sound of the task in which the transparent object is handled [see at least Paragraph 0030 for reference to the processor extracting features or characteristics of the object based on the obtained polarization information and/or depth information including the material of the object; Paragraph 0058 for reference to the feature extractor further identifying the acoustic properties of the identified material; Paragraph 0058 for reference to the feature extractor may refer to a table storing various properties of various materials including wood to identify the acoustic properties of the wood material; Paragraph 0060 for reference to any combination of such information being input into a machine learning system (e.g., a Convolutional Neural Network (“CNN”)) implemented in the feature extractor to extract features or characteristics of the object such as the shape, texture, surface roughness, material, etc.; Paragraph 0086 for reference to such polarization capture devices or systems being implemented in other systems for transparent objects detection]
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify the estimation of worker object detection of Zhang to include the transparent object detection of McEldowney. Doing so polarization information may be used to determine various properties of the object, as stated by McEldowney (Paragraph 0003).
Claim 8
While the combination of Zhang and McEldowney disclose the limitations above, regarding Claim 8, Zhang discloses the following:
wherein the data of the task sound includes data of a sound in an inaudible range [see at least Paragraph 0072 for reference to image and audio data being obtained from cameras installed or from existing camera and microphone systems; Paragraph 0078 for reference to the system including sensors to monitor the environment including microphones to detect ambient sound in an area, conversations of employees (e.g., clerks taking orders at the register), or other audio in the restaurant; Paragraph 0138 for reference to audio data being processed to detect unusual or undesirable conditions by detecting when music is played to loudly or too softly, when speech or environmental noise is too high, and so on; Figure 1 and related text regarding item 116 ‘audio data’]
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Jung, Minhyuk, and Seokho Chi. "Human activity classification based on sound recognition and residual convolutional neural network." Automation in Construction 114 (2020): 103177.
DOCUMENT ID
INVENTOR(S)
TITLE
US 2023/0109426 A1
Hashimoto et al.
Model generation apparatus, estimation apparatus, model generation method, and computer-readable storage medium storing a model generation program
US 2021/0116894 A1
Wichern et al.
MANUFACTURING AUTOMATION USING ACOUSTIC SEPARATION NEURAL NETWORK
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KRISTIN ELIZABETH GAVIN whose telephone number is (571)270-7019. The examiner can normally be reached M-F 7:30-4:30 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jerry O'Connor can be reached at 571-272-6787. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KRISTIN E GAVIN/Primary Examiner, Art Unit 3624