Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, 7-10, and 17-19 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Zalewski (US 2020/0118400 A1).
Regarding claims 1 and 17, Zalewski discloses a work estimation method that estimates a content of a work performed by a person, the work estimation method comprising:
obtaining first sound information and second sound information, the first sound information being related to a reflected sound that is a sound obtained by reflection of an emission sound in an inaudible frequency range [[0159] network of sensors for tracking user interaction with items provided for sale. Some embodiments include one or more of cameras, motions sensors, ultrasonic sensors, IR sensors, weight sensors, eye gaze sensors, face orientation sensors, gesture sensors, audio sensors, audio steering sensors, activity sensors, etc., which may be used to determine when a user reaches for an item and/or makes a take event (i.e., item was selected and possibly taken for purchase); [0339] very low frequency … pressure wave generated when the hammer strikes the element once is in one embodiment, reflected multiple times in both the element and the hammer, in accord with the elastic and acoustical properties of the ceramic and the hammer], the second sound information being related to a work sound generated by the work performed by the person [[0069] wireless coded communication devices includes the capability to detect or image the identity or attribute of a user, a biometric signature, a fingerprint, a voice, a sound; [0303] tracking and transmission of real-time power tool status, useful for example, to integrate into workplace safety and worker productivity tracking systems];
outputting image information that indicates a work area of the person, by inputting the first sound information obtained in the obtaining to a first trained model [[0052] movement is identified at least using image data from a camera. The camera is configured to produce image data that is processed as part of said sensor data that is received by said server; [0112] FIG. 28 illustrates an example of a vending machine, which may have a screen, selection input buttons, a slider for dispensing, and multiple sensors, in accordance with one embodiment.; [0148] FIG. 57 shows a machine learning system in a training phase, in accordance with one embodiment.];
outputting tool information that indicates a tool that is being used by the person, by inputting the second sound information obtained in the obtaining to a second trained model [[0290] wireless coded communication devices can be integrated into different objects, devices, structures or physical objects. When such devices move or are caused to move, a WCC device can be made trigger. Without limitation, example devices may include … tools, power tools, etc]; and
outputting work information that indicates the content of the work, by inputting, to a third trained model, the image information output in the outputting of the image information and the tool information output in the outputting of the tool information [[0303] tracking and transmission of real-time power tool status, useful for example, to integrate into workplace safety and worker productivity tracking systems].
Regarding claim 2, Zalewski teaches the work estimation method according to the work estimation method according to wherein the first trained model is a model trained using sound information related to the reflected sound and an image that indicates the work area of the person, the second trained model is a model trained using sound information related to the work sound and tool information that indicates a tool that is possibly used in the work, and the third trained model is a model trained using the image information, the tool information, and work content that indicates the content of the work [[0013] method includes receiving output of the sampling as feature inputs to a machine learning classifier model to derive one or more labels characterizing a state of an item; [0159] network of sensors for tracking user interaction with items provided for sale. Some embodiments include one or more of cameras, motions sensors, ultrasonic sensors, IR sensors, weight sensors, eye gaze sensors, face orientation sensors, gesture sensors, audio sensors, audio steering sensors, activity sensors, etc., which may be used to determine when a user reaches for an item and/or makes a take event (i.e., item was selected and possibly taken for purchase); [[0290] wireless coded communication devices can be integrated into different objects, devices, structures or physical objects. When such devices move or are caused to move, a WCC device can be made trigger. Without limitation, example devices may include … tools, power tools, etc].
Regarding claim 3, Zalewski teaches the work estimation method according to claim 1,wherein the first sound information includes at least one of a signal waveform of a sound or an image that indicates an arrival direction of the sound [[0159] audio steering sensors], and the second sound information includes a spectrogram image that indicates a frequency and power of the sound [[abstract] one or more sensors that include at least one camera capable of providing depth sensing to produce image data of a scene … one processing entity associated with the store detects the state of the item to change from one as item handled by said shopper; [0861] Kalman filtering applied to trajectory and location modelling. Non-parametric models and spectral methods may be used to capture the internal state of a user].
Regarding claim 4, Zalewski teaches the work estimation method according to The work estimation method according to wherein, in the outputting of the work information, the image information input to the third trained model includes a plurality of image frames [[0227] convolution neural network (CNN) to classify items that appear in images output from a camera located in the store. The model representing the CNN may be retrained when new products are added to the store].
Regarding claim 7, Zalewski teaches the work estimation method according to claim 1, further comprising: notifying the work information output in the outputting of the work information [[0303] a contractor can track all of his tool on multiple job sites, e.g., to determine where the tool is being used and the length of time or times the tool is used to perform a task. In one embodiment, a WCC on toolbox at job site notifies when (opened) tools are accessed. A WCC may be used for alerting a watch, cellphone or any endpoint the opening of a chest, toolbox, or any asset, including those assets on a job site, that the chest or toolbox was opened.].
Regarding claim 8, Zalewski teaches the work estimation method according to claim 7, further comprising: displaying the work information notified in the notifying [[0483] instructions sent back to the power tool 600 can include information to deactivate the tool, send an audible message, display message on a display device, and the like. The messages can be to notify the user of the power tool certain information. In one embodiment, the power tool does not include a display but messages are sent to a device such as a user's phone, watch, or to someone or something capable of relaying the message to a user, facility or management of the power tool].
Regarding claim 9, Zalewski teaches the work estimation method according to claim 1, the work estimation method according to wherein the outputting of the image information and the outputting of the tool information are executed when an output value of an acceleration sensor provided on a head of the person is lower than a predetermined threshold value [[0059] method further includes receiving, by the server, eye or head gaze information of the user. The eye or head gaze information is indicative of actions taken by the user with the item for which the user is interacting. The eye or head gaze information is collected to determine product information that is of interest to the user; [0303] tracking and transmission of real-time power tool status, useful for example, to integrate into workplace safety and worker productivity tracking systems; [0839] sense state … accelerometer].
Regarding claim 10, Zalewski teaches the work estimation method according to claim 1, further comprising: recording the work information output in the outputting of the work information, wherein, in the recording, when an output value of an acceleration sensor provided on a head of the person is higher than or equal to a predetermined threshold value, a time period during which the output value is higher than or equal to the predetermined threshold value is recorded as a non-work period [[0166] motion sensors can also be provided proximate to the shelves, or can be directed toward the shelves to determine when the item is being moved on the shelf or interactions occurring by an object, such as the hand of the user; [0201] detection of the user, the user's head orientation; [0303] tracking and transmission of real-time power tool status, useful for example, to integrate into workplace safety and worker productivity tracking systems. In one embodiment, it is possible to log the time of tool operation; [0739] detect motion of either the user's hand or hands interfacing with items or the items themselves moving on the shelf].
Regarding claim 18, Zalewski teaches the work estimation system according to claim 17, further comprising: an ultrasonic emitter that emits the emission sound; and a microphone that receives the reflected sound [[0184] ultrasonic sensors may also be integrated in various locations in the store, so as to identify when users or items are being moved, motion patterns, and other information; [0225] shelf units may contain or be monitored by one or more microphones to detect activity in connection with the items on the shelf. Commands and gestures may be detected using sensors.].
Regarding claim 19, Zalewski teaches a non-transitory computer-readable recording medium having recorded thereon a computer program for causing a computer to execute the work estimation method according to claim 1 [[0866] apparatus can be specially constructed for the required purpose, or the apparatus can be a general-purpose computer selectively activated or configured by a computer program stored in the computer. The apparatus may be housed with insulated housing to limit the volume of sound that may occur in some embodiments during an activation cycle. In particular, various general-purpose machines can be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations].
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Zalewski (US 2020/0118400 A1) as applied to claim 4 above, and further in view of Kalva (US 2008/0205515 A1).
Regarding claim 5, Zalewski does not explicitly teach and yet Kalva teaches the work estimation method according to claim 4, wherein in the outputting of the work information, a total number of image frames to be input to the third trained model is determined based on a difference in a total number of pixels in the work area between two image frames among the plurality of image frames, the two image frames being preceding and successive image frames in analysis target frames [[0032] machine learning; [0033] since the inter coding depends on the similarities between the current frame with the previous frame, a frame difference (block 505) can be used to characterize this similarity; [prior art claim 7] The method as defined by claim 1, wherein said statistical parameters for groups of pixels from frames of training video signals and input video signals are derived from differences of blocks of pixels of individual frames.].
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the invention to combine the power tool and user tracking as taught by Zalewski, with the identification of the statistical difference between pixels of individual frames as taught by Kalva so that frames which are similar may be input to the training video segments (Kalva) [[prior art claim 7]].
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Zalewski (US 2020/0118400 A1) as applied to claim 1 above, and further in view of Greenwald (US 2005/0177929 A1).
Regarding claim 11, Zalewski does not explicitly teach and yet Greenwald teaches the work estimation method according to claim 1, wherein the reflected sound is a sound reflected at a predetermined distance or less from a head of the person [[0121] sensing apparatus includes at least one proximity sensor 300 that detects an extent of the wearer's head 12 when the helmet 100 is positioned relatively close to the head 12. Alternatively, the proximity sensor 300 is calibrated to detect a different body part of the wearer, such as the wearer's shoulder region; [0124] as the helmet 100 is placed near the wearer's head 12, the head raises the capacitance of the sensor 340. When the capacitance reaches a specified threshold, the oscillator 346 activates; [0125] proximity sensor 300 is an ultrasonic sensor 360 that operates by emitting bursts of high-frequency sound waves that reflect or "echo" from the outer surface of the wearer's head; [prior art claim 16] The sports helmet of claim 12, wherein the sensor is an ultrasonic proximity sensor that emits high-frequency sound waves to detect the presence or absence of the object in the helmet.].
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the invention to combine the power tool and user tracking as taught by Zalewski, with helmet and ultrasonic proximity sensor as taught by Greenwald so that the user’s wearing or not wearing the helmet may be determined (Greenwald) [[prior art claim 16]].
Claims 14-15 is rejected under 35 U.S.C. 103 as being unpatentable over Zalewski (US 2020/0118400 A1) as applied to claim 1 above, and further in view of Moulton (US 2014/0266693 A1).
Regarding claim 14, Zalewski does not explicitly teach and yet Moulton teaches the work estimation method according to claim 1, comprising: changing an emission frequency of the emission sound in the inaudible frequency range according to information that indicates whether the person has been performing a same work for a certain time period or information that indicates whether the person has stopped performing the work for a certain time period among the work information output in the outputting of the work information [[abstract] a sensor device (110), mounted in an area (2000), for sensing a person in the area (2000); [0028] sensor device switches from operating in the second state to the first state in the event that the motion sensing device fails to sense motion for a threshold period of time whilst the sensor device operates in the second state; [0030] distance sensing device … one or more ultrasonic sensors; [0232] the monitoring device 120 can be configured to dynamically adjust the frequency that the respective sensor device(s) 110 perform a sensing activity based upon a historical analysis of received sensor signals. In particular, the monitoring device 120, can determine which sensor devices 120 of the network 109 are associated with transferring a larger than average proportion of the sensor signals 113 to the monitoring device 120, wherein the monitoring device 120 can configure the respective sensor device 110 to conduct a sensing activity at a higher than average frequency].
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the invention to combine the power tool and user tracking as taught by Zalewski, with the dynamically changing of frequency of ultrasonic sensors as taught by Moulton so that if motion is not sensed for a threshold period of time the frequency state may be changed to reattempt checking motion (Moulton) [[0232]].
Regarding claim 15, Zalewski does not explicitly teach and yet Moulton the work estimation method according to claim 14, the work estimation method according to comprising: outputting control information to an emission device that emits the emission sound in the inaudible frequency range, when it is determined based on the work information that the person has been performing the same work for the certain time period or the person has stopped performing the work for the certain time period, the control information being for reducing the emission frequency of the emission sound [[abstract][0028][0030][0232]].
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the invention to combine the power tool and user tracking as taught by Zalewski, with the dynamically changing of frequency of ultrasonic sensors as taught by Moulton so that if motion is not sensed for a threshold period of time the frequency state may be changed to reattempt checking motion (Moulton) [[0232]].
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Zalewski (US 2020/0118400 A1) as applied to claim 1 above, and further in view of Amigo (US 2010/0241464 A1).
Regarding claim 16, Zalewski does not explicitly teach and yet Amigo teaches the work estimation method according to claim 1, comprising: providing a notification that prompts the person to rest, when it is determined based on the work information that the person has been performing a same work beyond a predetermined time period [[0127] notifications to the injured employee and/or the case worker(s) includes immediate feedback upon detection of an undesired lifting behavior (e.g., warning the employee when he/she is about to engage in dangerous/unsafe lifting), summaries of lifting behavior/activities for a period of time (e.g., lifting behavior for the past eight hours), and/or notification of conformance with a recovery plan (e.g., whether the employee has or has not exceeded the maximum allowable lifts, and if so, by how much). In some embodiments, caregivers, case workers, or other relevant personnel can input the personalized activity/lifting thresholds by a user interface similar to user interface module 105 (FIG. 1)].
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the invention to combine the power tool and user tracking as taught by Zalewski, with warning of exceeding activity thresholds as taught by Amigo so that workers are warned prior to being injured due to excessive work (Amigo) [[0127]].
Allowable Subject Matter
Claims 6 and 12-13 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: (see below).
Regarding claim 6, the closest prior art of record does not appear to teach the work estimation method according to claim 4, further comprising: selecting an image frame to be re-input to the third trained model from among the plurality of image frames, when the work information output in the outputting of the work information does not match any of the work information used when training the third trained model, wherein, in the selecting, two or more image frames are selected, the two or more image frames having a difference in a total number of pixels in the work area between two image frames among the plurality of image frames, the difference being lower than a predetermined threshold value, the two image frames being preceding and successive image frames in analysis target frames, and in the outputting of the work information, the two or more image frames selected in the selecting are re-input to the third trained model to output the work information that is in accordance with the re-input.
Regarding claim 12, Zalewski teaches [[0052] movement is identified at least using image data from a camera. The camera is configured to produce image data that is processed as part of said sensor data that is received by said server.; [0053] a confidence level of identifying said movement is increased by using input from one or more other sensors, each sensor is provided a weighting that gives a respective sensor or sensors more or less significance in said identifying of the movement.]. However, Zalewski does not appear to teach the work estimation method according to claim 1, the work estimation method according to wherein the outputting of the work information includes changing a weight applied to the image information to be input to the third trained model according to a rate of change in preceding and successive reflection waveforms in analysis target frames among reflection waveforms of the reflected sound included in the first sound information.
Regarding claim 13, Zalewski teaches [[0052][0053]]. However, Zalewski does not appear to teach the work estimation method according to claim 1, comprising: comparing reflection waveforms of the reflected sound included in the first sound information, wherein, when it is determined in the comparing that a rate of change in preceding and successive reflection waveforms in analysis target frames is higher than or equal to a predetermined threshold value, the outputting of the work information includes setting a weight applied to the image information to be input to the third trained model to be lower than a weight applied to the tool information.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN D ARMSTRONG whose telephone number is (571)270-7339. The examiner can normally be reached M - F 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Isam Alsomiri can be reached at 571-272-6970. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN D ARMSTRONG/ Examiner, Art Unit 3645
Prosecution Insights