CTNF 18/954,808 CTNF 101489 DETAILED ACTION Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1-2 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Hirotaka (JP 2023016494 A) in view of Sagawa (JP 7165108 B2) . As per claim 1, Hirotaka teaches the claimed: 1. An information processing method, performed by a processor included in a controller, (Hirotaka (page 7, line 4-5): “the controller 15 may be realized by a process executed by a hardware processor.”) for identifying a position that a worker visually recognizes with carefulness, comprising: acquiring as image data an image representing at least a part of a field of view of the worker at a predetermined time; acquiring as line-of-sight data a line-of-sight direction of the worker at the predetermined time; acquiring as electroencephalogram signal data an electroencephalogram signal of the worker in a time interval containing the predetermined time; (Hirotaka (page 7, line 7-16): “The acquisition unit 15A is a processing unit that acquires electroencephalogram data of the worker … The acquisition unit 15A may acquire the electroencephalogram data in real time , or may acquire the data in batch processing.” Hirotaka (page 5, line 26-28): “The term "electroencephalogram data" as used herein refers to waveforms of electrical signals corresponding to electroencephalograms, for example, time-series data of signal intensity .” Hirotaka teaches the times series data, which can correspond to the predetermined time interval explained below with reference Sagawa, and the predetermined time can be any time within the range of the time interval.) detecting a predetermined waveform from the acquired electroencephalogram signal data; and (Hirotaka (Abstract): “an acquisition unit for acquiring brain wave data of a first worker to evaluate for each work for operating a plant and second brain wave data to base on when evaluating the first worker” Hirotaka (page 4, line 4-6): “… electroencephalogram data including changes in the electroencephalogram waveform corresponding to the perception and recognition of the event of the operation performed on the operating device is measured.” Hirotaka teaches the electroencephalogram data being able to detect changes in the waveform, and this change in waveform correspond to the predetermined waveform since any change deviating from an original waveform is detected and measured.) Hirotaka alone does not explicitly teach the remaining claim limitations. However, Hirotaka in combination with Sagawa teaches the claimed: acquiring as image data an image representing at least a part of a field of view of the worker at a predetermined time; (Sagawa (page 2, line 27-31): “The input device 20 is a device for inputting information to the information processing device 10, and includes a camera 21 for acquiring an image (line-of-sight image) in the line-of-sight direction of the worker (trainee), and a line-of-sight image of the trainee with respect to the line-of-sight image. and a line-of-sight position acquisition device 22 having a function of acquiring data indicating a position (line-of-sight position data).” Sagawa (page 5, line 13-14): “line-of-sight video and line-of-sight position data are acquired at predetermined time intervals ”) acquiring as line-of-sight data a line-of-sight direction of the worker at the predetermined time; (Sagawa (page 2, line 27-31): “The input device 20 is a device for inputting information to the information processing device 10, and includes a camera 21 for acquiring an image (line-of-sight image) in the line-of-sight direction of the worker (trainee), and a line-of-sight image of the trainee with respect to the line-of-sight image. and a line-of-sight position acquisition device 22 having a function of acquiring data indicating a position (line-of-sight position data).” Sagawa (page 5, line 13-14): “line-of-sight video and line-of-sight position data are acquired at predetermined time intervals ”) determining a visually recognized position by the worker when the predetermined waveform occurs, based on the image data and the line-of-sight data. (Hirotaka (page 5, line 26-28): “The term "electroencephalogram data" as used herein refers to waveforms of electrical signals corresponding to electroencephalograms, for example, time-series data of signal intensity .” Sagawa (page 9, line 8-10): “… the reference time-series line-of-sight transition map 143 is a time-series line-of-sight transition map generated in advance from the line-of-sight video and line-of-sight position data in the model work of the expert .” Sagawa teaches the line-of-sight video and line of sight position data, which correspond to the image data and the line-of-sight data. Sagawa also teaches the line-of-sight transition map, which is the visually recognized position of the worker. Combining the time series data of the signal intensity and the time series data of the line-of-sight transition map can produce a specific visually recognized position when the predetermined waveform occurs and matching it up with the generated line-of-sight transition map from the image data and position data.) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the line-of-sight data as taught by Sagawa with the system of Hirtaka in order to visually analyze and determine the position and gaze of a worker. As per claim 2, Hirotaka teaches the claimed: 2. The information processing method according to claim 1, wherein detecting the predetermined waveform includes detecting the predetermined waveform when a part of the electroencephalogram signal has a waveform corresponding to a reference waveform. (Hirotaka (page 4, line 15-16): “as an example of the electroencephalogram data of the second worker to be used as a reference ” Hirotaka (page 5, line 34 – page 6, line 1-2): “the electroencephalogram data of the new worker 3A can be set as an evaluation target, while the electroencephalogram data of the experienced worker 3B can be set as a reference .” Hirotaka (page 4, line 4-6): “… electroencephalogram data including changes in the electroencephalogram waveform corresponding to the perception and recognition of the event of the operation performed on the operating device is measured.” Hirotaka teaches the electroencephalogram data being able to detect changes in the waveform, and this change in waveform correspond to the predetermined waveform since any change deviating from an original waveform is detected and measured. Thus, using another worker’s electroencephalogram data as a reference, it will be able to determine whether or not it corresponds to the reference waveform when detecting a predetermined waveform.) As per claim 7, the reasons and rationale for the rejection of claim 1 is incorporated herein. In particular, only additional features unique to claim 7 that were not present in claim 1 will be explicitly addressed here. Hirotaka teaches the claimed: (Strikethrough taught in claim 1) 7. A system comprising: an imaging device configured to acquire as image data an image representing at least a part of a field of view of a worker at a predetermined time; a line-of-sight identifier configured to acquire as line-of-sight data a line-of-sight direction of the worker at the predetermined time; an electroencephalograph configured to acquire as electroencephalogram signal data an electroencephalogram signal of the worker in a time interval containing the predetermined time; and a controller including a storage configured to store the image data acquired from the imaging device, the line-of-sight data acquired from the line-of-sight identifier, and the electroencephalogram signal data acquired from the electroencephalograph, and a processor configured to execute instructions to detect a predetermined waveform from the electroencephalogram signal data and determining a visually recognized position by the worker when the predetermined waveform occurs, based on the image data and the line-of-sight data . (Sagawa (page 4, line 5-10): “the data storage unit 14 stores line-of-sight video position data … the line-of-sight video position data 141 is information in which the line-of-sight video acquired by the camera 21 and the line-of-sight position data acquired by the line-of-sight position acquisition device 22 are combined.” Hirotaka (page 5, line 20-21): “the storage unit 13 stores first electroencephalogram data 13A and second electroencephalogram data 13B.”) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the line-of-sight data as taught by Sagawa with the system of Hirtaka in order to visually analyze and determine the position and gaze of a worker . 07-21-aia AIA Claim s 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Hirotaka in view of Sagawa in further view of Ke (“Monitoring Distraction of Construction Workers Using a Wearable Electroencephalography (EEG) Device”, 2019) . As per claim 3, Hirotaka and Sagawa alone does not explicitly teach the claimed limitations. However, Hirotaka and Sagawa in combination with Ke teaches the claimed: 3. The information processing method according to claim 2, wherein the reference waveform is defined based on a specific electroencephalogram signal measured in advance that is induced when the worker views a predetermined event. (Ke (Abstract): “sustained attention to response task (SART) has been employed to induce the occurrences of distraction in the simulated construction safety inspection tasks . The recorded EEG data was divided into two groups corresponding with task performance: focused and distracted.” Ke teaches the simulated tasks, which correspond to a predetermined event. Hirotaka (page 4, line 4-6): “… electroencephalogram data including changes in the electroencephalogram waveform corresponding to the perception and recognition of the event of the operation performed on the operating device is measured.” Hirotaka teaches being able to determine and measure electroencephalogram data corresponding to the recognition of an event. This electroencephalogram data can be said to be a reference waveform and is measured in advance because it is from the first worker and, with the simulated tasks of Ke, induced when the worker views a predetermined event.) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the simulated tasks as taught by Ke with the system of Hirotaka as modified by Sagawa in order to determine a baseline for the waveforms of workers when encountering a specific event. As per claim 4, Hirotaka and Sagawa alone does not explicitly teach the claimed limitations. However, Hirotaka and Sagawa in combination with Ke teaches the claimed: 4. The information processing method according to claim 3, wherein the predetermined event is an event that attracts the worker's attention. (Ke (Abstract): “sustained attention to response task (SART) has been employed to induce the occurrences of distraction in the simulated construction safety inspection tasks . The recorded EEG data was divided into two groups corresponding with task performance: focused and distracted.” Hirotaka (page 9, line 4-5): “When detecting perceptual changes and cognitive changes based on event-related potentials in this way”) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the simulated tasks as taught by Ke with the system of Hirotaka as modified by Sagawa in order to determine a baseline for the waveforms of workers when encountering a specific event . 07-21-aia AIA Claim s 5-6 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Hirotaka in view of Sagawa in further view of Gould (US 20200134877 A1) . As per claim 5, Hirotaka and Sagawa alone does not explicitly teach the claimed limitations. However, Hirotaka and Sagawa in combination with Gould teaches the claimed: 5. The information processing method according to claim 1, further comprising: conveying the visually recognized position to another worker different from the worker. (Gould [0009]: “the special points are communicated to another worker with an AR headset.” Gould [0053]: “The points can be saved to the system, and another worker can walk through the rooms and perform the work identified by the supervisor by relying on the local artificial frame(s) of reference.”) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the artificial frames of reference as taught by Gould with the system of Hirotaka as modified by Sagawa in order to allow users to view and utilize the augmented image for faster and efficient analysis and communication with each other. As per claim 6, Hirotaka and Sagawa alone does not explicitly teach the claimed limitations. However, Hirotaka and Sagawa in combination with Gould teaches the claimed: 6. The information processing method according to claim 1, further comprising: imparting a visual effect representing the visually recognized position onto the image. (Gould [0009]: “The artificial frame of reference and the special points are communicated to another worker with an AR headset. The AR headset of the second worker displays an augmented image that shows the special points indicated against the wall. Thus, the second worker can visualize the special points ” Gould [0053]: “The points can be saved to the system, and another worker can walk through the rooms and perform the work identified by the supervisor by relying on the local artificial frame(s) of reference .”) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the artificial frames of reference as taught by Gould with the system of Hirotaka as modified by Sagawa in order to allow users to view and utilize the augmented image for faster and efficient analysis and communication with each other. As per claim 8, Hirotaka and Sagawa alone does not explicitly teach the claimed limitations. However, Hirotaka and Sagawa in combination with Gould teaches the claimed: 8. The system according to claim 7, further comprising: a display configured to display the image data and effect-imparted image data generated based on visually recognized position data representing the visually recognized position. (Gould [0006]: “An augmented reality (AR) headset receives the augmented data and generates an augmented image to display to the worker wearing the AR headset . The augmented image includes a combination of the physical image in front of the AR headset and the worker, and a virtual image of objects from a construction plan. Images of the virtual objects are placed at locations in the augmented image that correspond to where the corresponding physical objects should be placed according to the construction plan.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the artificial frames of reference as taught by Gould with the system of Hirotaka as modified by Sagawa in order to allow users to view and utilize the augmented image for faster and efficient analysis and communication with each other. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSHUA SUO whose telephone number is (571) 272-8387. The examiner can normally be reached Mon-Fri 8am-5pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached on (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /JOSHUA SUO/Examiner, Art Unit 2616 /DANIEL F HAJNIK/Supervisory Patent Examiner, Art Unit 2616 Application/Control Number: 18/954,808 Page 2 Art Unit: 2616 Application/Control Number: 18/954,808 Page 3 Art Unit: 2616 Application/Control Number: 18/954,808 Page 4 Art Unit: 2616
Prosecution Insights