DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
1. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/25/2025 has been entered.
Response to Amendment
2. Claims 1 and 4 are currently pending.
3. Claims 2-3 and 5-6 are canceled.
4. Claims 1 and 4 are currently amended.
5. The 112(b) rejections to claims have been overcome.
Claim Rejections - 35 USC § 103
6. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
8. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
9. Claims 1 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Taveira (US 20190310629 A1), in view of Kim (US 20150269427 A1), and in further view of Tamagaki (WO 2016157883 A1).
10. Regarding Claim 1, Taveira teaches a remote operation assistance system that is used for assisting in remote operation of a work machine by an operator using a remote operation device via a network, by using the network connecting the work machine, the remote operation device for remotely operating the work machine, and a server, the remote operation assistance system comprising (Taveira [0063], [0066], and [0076]):
Wherein the remote operation device includes: a remote operation mechanism configured to be operated by the operator to drive the work machine, and transmit a signal according to an operation manner to the work machine (Taveira: [0025] and [0076]),
A distance detection device arranged at a predetermined position of the remote operation device… and an image capturing device configured to capture an image around the operator (Taveira: [0029] and [0038] Note that determining a presence of other individuals in the environment of the operator is equivalent to detecting a distance from the operator to the moving body.),
The work machine includes a driving mechanism configured to receive the signal according to the operation manner of the remote operation mechanism and drive the work machine based on the signal that is received (Taveira: [0024] and [0067]),
The server includes a remote-control device composed of a processor, and the remote-control device includes: a distance acquisition unit composed of the processor and configured to obtain information on the distance… detected by the distance detection device (Taveira: [0026], [0029], and [0038]),
An attribute acquisition unit composed of the processor and configured to recognize... the moving body contained in a captured image captured by the image capturing device (Taveira: [0038] and [0053]),
An interference determination unit composed of the processor and configured to determine whether or not an index value related to a level of a likelihood of interference occurring with the remote operation reaches a predetermined danger threshold, based on the distance detected by the distance acquisition unit and the attribute obtained by the attribute acquisition unit (Taveira: [0035], [0042], and [0044] Note that the attention level is equivalent to the likelihood of interference occurring with the remote operation. Also, note that indicating the operator is below a desired threshold is equivalent to the remote operation reaching a predetermined danger threshold.),
By determining that, as the distance is shorter, there is a higher likelihood of interference with the remote operation, and determining that, as the activity level in the attribute is higher, there is a higher likelihood of interference with the remote operation (Taveira: [0029], [0036], and [0038] Note that determining an increase in the chance of the attention level falls below an acceptable level is equivalent to determining there is a higher likelihood of interference with remote operation. Also, determining a presence of an individual is equivalent to a distance being shorter (e.g., a presence of an individual is a short distance and no presence of an individual is a larger distance), and determining an activity level is high is equivalent to a presence of sounds or events (conversation with nearby person).),
And an assistance control unit composed of the processor and configured to perform control to assist in the remote operation by limiting or stopping transmission of the signal according to the operation manner in a case in which the interference determination unit determines that the index value reaches the danger threshold (Taveira: [0057]),
And the assistance control unit performs control to decelerate motion of the work machine in response to the interference determination unit determining... the level of the likelihood of interference occurring with the remote operation performed by the operator... (Taveira: [0056] Note that automatic control of the robot vehicle is executed in response to a likelihood of interference with the remote operation performed by the operator occurring. Also, note that restricting speed is equivalent to decelerating motion of the work machine.).
Taveira fails to explicitly teach an attribute database composed of a storage device provided in the remote operation device or the server and configured to store a candidate as a moving body and an attribute to which the moving body belongs and which includes information on an activity level of the moving body; a distance acquisition unit composed of the processor and configured to obtain information on the distance from the operator to the moving body…; and an attribute acquisition unit... configured to recognize a shape of the moving body... by performing image analysis of the captured image, extract a candidate as the moving body having a shape that is similar to the recognized shape of the moving body from among a plurality of the candidates as the moving body stored beforehand in the attribute database, and refer to an attribute to which the moving body that is extracted belongs.
However, in the same field of endeavor, Kim teaches an attribute database composed of a storage device provided in the remote operation device or the server and configured to store a candidate as a moving body and an attribute to which the moving body belongs and which includes information on an activity level of the moving body (Kim: [0007] and [0029] Note that storing a candidate as a moving body is equivalent to the processor using databases to identify humans. Also, storing an attribute to which the moving body belongs including information on an activity level is equivalent to detecting/identifying the pose of the human.),
A distance acquisition unit composed of the processor and configured to obtain information on the distance from the operator to the moving body, detected by the distance detection device (Kim: [0075] and [0076] Note that the information on the distance from the operator to the moving body is equivalent to the distance between the person and detection zone determined by the modules. The zones are distance thresholds to determine the proximity of the person in the environment.),
And an attribute acquisition unit composed of the processor and configured to recognize a shape of the moving body contained in a captured image captured by the image capturing device by performing image analysis of the captured image, extract a candidate as the moving body having a shape that is similar to the recognized shape of the moving body from among a plurality of the candidates as the moving body stored beforehand in the attribute database, and refer to an attribute to which the moving body that is extracted belongs (Kim: [0054], [0055], and [0057]).
Taveira and Kim are considered to be analogous to the claim invention because they are in the same field of object recognition. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Taveira to incorporate the teachings of Kim to include an attribute database that stores a candidate as a moving body and an attribute including an activity level for extracting a candidate as the moving body having a shape similar to a recognized shape and obtain a distance from the operator to the moving body because it provides the benefit of accurately identifying objects in a surrounding environment and improving productivity in a workspace.
Taveira and Kim fail to explicitly teach the assistance control unit performs control to stop motion of the work machine in response to the interference determination unit determining that the level of the likelihood of interference occurring with the remote operation performed by the operator is equal to or higher than a predetermined value, and the assistance control unit performs control... in response to the interference determination unit determining that the level of the likelihood of interference occurring with the remote operation performed by the operator is lower than the predetermined value.
However, in the same field of endeavor, Tamagaki teaches the assistance control unit performs control to stop motion of the work machine in response to the interference determination unit determining that the level of the likelihood of interference occurring with the remote operation performed by the operator is equal to or higher than a predetermined value (Tamagaki: [Page 7, Paragraph 2] and [Page 9, Paragraph 3] Note that determining that the level of the likelihood of interference occurring with the operation performed by the operator being higher than a predetermined value is equivalent to the ECU 70 determining an abnormal state of based on the second threshold. Also, note that the interference occurring is equivalent to the operator of the vehicle being in an abnormal state (inattention state, see [Page 7, Paragraph 8]).),
And the assistance control unit performs control... in response to the interference determination unit determining that the level of the likelihood of interference occurring with the remote operation performed by the operator is lower than the predetermined value (Tamagaki: [Page 7, Paragraph 7] and [Page 9, Paragraph 2] Note that determining that the level of the likelihood of interference occurring with the operation performed by the operator being lower than a predetermined value is equivalent to the provisional determination of the operator abnormality. The predetermined value is equivalent to the second threshold time. Also, note that the interference occurring is equivalent to the operator of the vehicle being in an abnormal state/provisional abnormal state (inattention state, see [Page 7, Paragraph 8]).).
Tamagaki discloses this claimed limitation except for explicitly teaching to decelerate motion of the work machine. It would have been an obvious matter of design choice to decelerate motion, since the applicant has not disclosed anything that solves any stated problem or is for any particular purpose and it appears that the invention would perform equally as well performing another mitigating action. For example, Taveira explains in [0056] to perform automatic control including restricting the speed and/or maneuverability of the vehicle in response to the likelihood of interference. Therefore, it would have been obvious to one of ordinary skill in the art to substitute the Lane Keep Assist control with restricting the speed [decelerate motion] and maneuverability (LKA).
Taveira, Kim, and Tamagaki are considered to be analogous to the claim invention because they are in the same field of vehicle control and object recognition. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Taveira and Kim to incorporate the teachings of Tamagaki to stop the motion of the work machine in response to a high likelihood of interference with the remote operation performed by the operator and to provide another control in response to a low likelihood of interference with the remote operation performed by the operator. This is because performing different levels of control based on different level of attentiveness provide the benefit of improving the interference likelihood determination and minimizes false detections.
11. Regarding Claim 4, Taveira teaches a remote operation assistance system that is used for assisting in remote operation of a work machine by an operator using a remote operation device via a network, by using the network connecting the work machine, and the remote operation device for remotely operating the work machine, wherein the remote operation device includes (Taveira [0063], [0066], and [0076]):
A remote operation mechanism configured to be operated by the operator to drive the work machine, and transmit a signal according to an operation manner to the work machine (Taveira: [0025] and [0076]),
A distance detection device arranged at a predetermined position of the remote operation device…. and an image capturing device configured to capture an image around the operator (Taveira: [0029] and [0038] Note that determining a presence of other individuals in the environment of the operator is equivalent to detecting a distance from the operator to the moving body.),
And a remote-control device composed of a processor, the work machine includes a driving mechanism configured to receive the signal according to the operation manner of the remote operation mechanism and drive the work machine based on the signal that is received, and the remote-control device includes (Taveira: [0024] and [0067]):
A distance acquisition unit composed of the processor and configured to obtain information on the distance… detected by the distance detection device (Taveira: [0026], [0029], and [0038]),
An attribute acquisition unit composed of the processor and configured to recognize... the moving body contained in a captured image captured by the image capturing device (Taveira: [0038] and [0053]),
An interference determination unit composed of the processor and configured to determine whether or not an index value related to a level of a likelihood of interference occurring with the remote operation reaches a predetermined danger threshold, based on the distance detected by the distance acquisition unit and the attribute obtained by the attribute acquisition unit (Taveira: [0035], [0042], and [0044] Note that the attention level is equivalent to the likelihood of interference occurring with the remote operation. Also, note that indicating the operator is below a desired threshold is equivalent to the remote operation reaching a predetermined danger threshold.),
By determining that, as the distance is shorter, there is a higher likelihood of interference with the remote operation, and determining that, as the activity level in the attribute is higher, there is a higher likelihood of interference with the remote operation (Taveira: [0029], [0036], and [0038] Note that determining an increase in the chance of the attention level falls below an acceptable level is equivalent to determining there is a higher likelihood of interference with remote operation. Also, determining a presence of an individual is equivalent to a distance being shorter (e.g., a presence of an individual is a short distance and no presence of an individual is a larger distance), and determining an activity level is high is equivalent to a presence of sounds or events (conversation with nearby person).),
And an assistance control unit composed of the processor and configured to perform control to assist in the remote operation by limiting or stopping transmission of the signal according to the operation manner in a case in which the interference determination unit determines that the index value reaches the danger threshold (Taveira: [0057]).
And the assistance control unit performs control to decelerate motion of the work machine in response to the interference determination unit determining... the level of the likelihood of interference occurring with the remote operation performed by the operator... (Taveira: [0056] Note that automatic control of the robot vehicle is executed in response to a likelihood of interference with the remote operation performed by the operator occurring. Also, note that restricting speed is equivalent to decelerating motion of the work machine.).
Taveira fails to explicitly teach an attribute database composed of a storage device configured to store a candidate as the moving body and an attribute to which the moving body belongs and which includes information on an activity level of the moving body; a distance acquisition unit… configured to obtain information on the distance from the operator to the moving body, detected by the distance detection device, and an attribute acquisition unit... configured to recognize a shape of the moving body... by performing image analysis of the captured image, extract a candidate as the moving body having a shape that is similar to the recognized shape of the moving body from among a plurality of the candidates as the moving body stored beforehand in the attribute database, and refer to an attribute to which the moving body that is extracted belongs.
However, in the same field of endeavor, Kim teaches an attribute database composed of a storage device configured to store a candidate as the moving body and an attribute to which the moving body belongs and which includes information on an activity level of the moving body (Kim: [0007] and [0029] Note that storing a candidate as a moving body is equivalent to the processor using databases to identify humans. Also, storing an attribute to which the moving body belongs including information on an activity level is equivalent to detecting/identifying the pose of the human.),
A distance acquisition unit composed of the processor and configured to obtain information on the distance from the operator to the moving body, detected by the distance detection device (Kim: [0075] and [0076] Note that the information on the distance from the operator to the moving body is equivalent to the distance between the person and detection zone determined by the modules. The zones are distance thresholds to determine the proximity of the person in the environment.),
And an attribute acquisition unit composed of the processor and configured to recognize a shape of the moving body contained in a captured image captured by the image capturing device by performing image analysis of the captured image, extract a candidate as the moving body having a shape that is similar to the recognized shape of the moving body from among a plurality of the candidates as the moving body stored beforehand in the attribute database, and refer to an attribute to which the moving body that is extracted belongs (Kim: [0054], [0055], and [0057]).
Taveira and Kim are considered to be analogous to the claim invention because they are in the same field of object recognition. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Taveira to incorporate the teachings of Kim to include an attribute database that stores a candidate as a moving body and an attribute including an activity level for extracting a candidate as the moving body having a shape similar to a recognized shape and obtain a distance from the operator to the moving body because it provides the benefit of accurately identifying objects in a surrounding environment and improving productivity in a workspace.
Taveira and Kim fail to explicitly teach the assistance control unit performs control to stop motion of the work machine in response to the interference determination unit determining that the level of the likelihood of interference occurring with the remote operation performed by the operator is equal to or higher than a predetermined value, and the assistance control unit performs control... in response to the interference determination unit determining that the level of the likelihood of interference occurring with the remote operation performed by the operator is lower than the predetermined value.
However, in the same field of endeavor, Tamagaki teaches the assistance control unit performs control to stop motion of the work machine in response to the interference determination unit determining that the level of the likelihood of interference occurring with the remote operation performed by the operator is equal to or higher than a predetermined value (Tamagaki: [Page 7, Paragraph 2] and [Page 9, Paragraph 3] Note that determining that the level of the likelihood of interference occurring with the operation performed by the operator being higher than a predetermined value is equivalent to the ECU 70 determining an abnormal state of based on the second threshold. Also, note that the interference occurring is equivalent to the operator of the vehicle being in an abnormal state (inattention state, see [Page 7, Paragraph 8]).),
And the assistance control unit performs control... in response to the interference determination unit determining that the level of the likelihood of interference occurring with the remote operation performed by the operator is lower than the predetermined value (Tamagaki: [Page 7, Paragraph 7] and [Page 9, Paragraph 2] Note that determining that the level of the likelihood of interference occurring with the operation performed by the operator being lower than a predetermined value is equivalent to the provisional determination of the operator abnormality. The predetermined value is equivalent to the second threshold time. Also, note that the interference occurring is equivalent to the operator of the vehicle being in an abnormal state/provisional abnormal state (inattention state, see [Page 7, Paragraph 8]).).
Tamagaki discloses this claimed limitation except for explicitly teaching to decelerate motion of the work machine. It would have been an obvious matter of design choice to decelerate motion, since the applicant has not disclosed anything that solves any stated problem or is for any particular purpose and it appears that the invention would perform equally as well performing another mitigating action. For example, Taveira explains in [0056] to perform automatic control including restricting the speed and/or maneuverability of the vehicle in response to the likelihood of interference. Therefore, it would have been obvious to one of ordinary skill in the art to substitute the Lane Keep Assist control with restricting the speed [decelerate motion] and maneuverability (LKA).
Taveira, Kim, and Tamagaki are considered to be analogous to the claim invention because they are in the same field of vehicle control and object recognition. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Taveira and Kim to incorporate the teachings of Tamagaki to stop the motion of the work machine in response to a high likelihood of interference with the remote operation performed by the operator and to provide another control in response to a low likelihood of interference with the remote operation performed by the operator. This is because performing different levels of control based on different level of attentiveness provide the benefit of improving the interference likelihood determination and minimizes false detections.
Response to Arguments
12. Applicant's arguments filed 3/25/2025 have been fully considered but they are not persuasive.
13. The Applicant has alleged that "when calculating the attention level, the attribute of the moving object around the operator is not considered" and "the distance between the moving object and the operator is not considered when calculating the attention level" regarding Taveira. The Examiner disagrees.
As currently claimed, the attribute of the moving object may be broadly interpreted as a pose/position that indicates an action of the moving body. In Taveira, [0029] and [0036] teaches that the presence of sounds, events, and individuals may distract the driver. This determination is equivalent to determining there is a higher likelihood of interference with the remote operation. More specifically, Taveira determines sounds and a nearby person to determine the operator is conversating with the moving body. Therefore, under the broadest reasonable interpretation, the activity level in the attribute is interpreted as the moving body conversating with the operator. There is no indication in the claims that further defines the types of activities that indicate a higher likelihood of interference with the operator.
In addition, determining that as distance between the operator and moving body is shorter, the higher the likelihood of interference with the remote operation may be broadly interpreted as determining a presence of another individual. Taveira teaches in [0029] that a presence of another individual may distract the operator. This is equivalent to determining as the distance is shorter, there is a higher likelihood of interference. The presence of an individual increases the likelihood, and therefore, the absence of an individual decreases the likelihood. An absence of an individual in the environment indicates a moving body would be at a much greater distance from the operator. Under the broadest reasonable interpretation, the distance being shorter resulting in a higher likelihood of interference is equivalent to a presence of an individual in the environment of the remote operator possibly distracting the operator.
14. All other arguments provided by the Applicant are moot because Kim (US 20150269427 A1) and Tamagaki (WO 2016157883 A1) have been applied to teach the remaining limitations with arguments.
15. As a result, Taveira (US 20190310629 A1), in view of Kim (US 20150269427 A1), and in further view of Tamagaki (WO 2016157883 A1) teaches all aspects of the invention. The rejection is modified according to the newly amended language but still maintained with the current prior art of record.
16. Claims 1 and 4 remain rejected under their respective grounds and rational as cited above, and as stated in the prior office action which is incorporated herein. Also, although not specifically argued, all remaining claims remain rejected under their respective grounds, rationales, and applicable prior art for these reasons cited above, and those mentioned in the prior office action which is incorporated herein.
Prior Art
17. The prior art made of record and not relied upon is considered most relevant to applicant's disclosure.
Gurusubramanian (US 20190225228 A1)
Sim (US 20200122734 A1)
Tamagaki (US 20200148214 A1)
Victor (US 20190367050 A1)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL T SILVA whose telephone number is (571)272-6506. The examiner can normally be reached Mon-Tues: 7AM - 4:30PM ET; Wed-Thurs: 7AM-6PM ET; Fri: OFF.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Angela Ortiz can be reached on 571-272-1206. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL T SILVA/Examiner, Art Unit 3663
/ANGELA Y ORTIZ/Supervisory Patent Examiner, Art Unit 3663