CTNF 19/045,174 CTNF 96565 DETAILED ACTION Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. Joint Inventors 07-20-02-aia AIA This application currently names joint inventors. In considering patentability of the claims, the Examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the Examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. Information Disclosure Statement The information disclosure statements (IDS) submitted on 02/04/2025, 02/11/2025, and 10/10/2025, were filed before the mailing of a First Office Action on the Merits. The submissions are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner. Priority Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55. The current application claims foreign priority to JP2024-028399 which has an earlier filed date of 02/28/2024. Examiner has checked and verified that the subject matter of the instant application is supported by the foreign document, and as such, the earlier filed date of 02/28/2024 is granted. 12-151 AIA 26-51 12-51 Status of Claims This action is in response to Applicant’s filing on 02/04/2025. Claims 1-9 are pending and examined below. Claim Rejections - 35 USC § 101 Examiner notes that the current independent claims are not rejected under 35 U.S.C. 101 because they recite a homography process which is not something that can reasonably be performed in the human mind. This process requires numerous matrix conversions of individual pixels of an image which can, depending on the resolution of the image, be in the millions to hundreds of millions (megapixel scale, etc.) which is simply not something that can be reasonably performed in the human mind. Claim Rejections - 35 USC § 112 07-30-02 AIA The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. Claim 3 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 3 recites the limitation of “dynamically select one corresponding to the gaze direction among the plurality of cameras as the reference camera”, however, the limitation is indefinite for two reasons. First, the phrase “dynamically select” is not sufficiently described in the specification as it makes no mention of how the selection is “dynamic”. Second, the limitation references selecting “one”, however, it is not clear what is actually being selected due to the awkward language. For example, it is unclear if a gaze direction is being selected or if a camera of the plurality of cameras is being selected. For the purposes of examination, the term “one” is being interpreted to cover a camera of the plurality of cameras. Claim Rejections - 35 USC § 102 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-12-aia AIA (a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention. Claims 1 and 5-9 are rejected under both 35 U.S.C. 102(a)(1) and 35 U.S.C. 102(a)(2) for being anticipated by Kelly et al., US 20110066262 A1, herein referred to as Kelly. Regarding claim 1 , Kelly discloses the following: A remote operator terminal used by a remote operator for a remote operation of a moving body (Figs. 4, 6-7, Paragraph 0047) The remote system may be used for remote operation of a vehicle a first image is an image captured at a first timing by a camera mounted on the moving body (Figs. 4, 6-7, Paragraphs 0047-0052) cameras may acquire data in real-time cameras may be attached to the remote apparatus which can be a vehicle this data being acquired can be considered as being acquired at a first timing a first point of view is defined by a combination of a position and a viewing direction of the camera at the first timing (Figs. 4, 6-7, Paragraphs 0047-0052) each camera may have a given field of view based on its current position and viewing direction a second point of view is defined by a combination of a predicted position and a predicted viewing direction of the camera at a second timing later than the first timing (Figs. 4, 6-7, Paragraphs 0047-0052, 0066-0069) a synthetic point of view may be created through a predicted camera points of view this prediction may be based on positioning, orientation, etc. a homography process converts the first image viewed from the first point of view into a second image viewed from the second point of view based on perspective projection transformation using a projection plane associated with the camera (Figs. 4, 6-7, Paragraphs 0047-0052, 0066-0070) a homography process may be used to produce a synthetic image frame this second image frame has a second point of view examiner notes that homography is a term of art and is well defined as utilizing a common projection plane between two perspectives acquire a plurality of first images respectively captured at the first timing by a plurality of cameras mounted on the moving body (Figs. 4, 6-7, Paragraphs 0047-0052, 0066-0070) multiple cameras may be used for capturing images at a first time acquire a combination of a plurality of second images by applying the homography process using a common projection plane to a combination of the plurality of first images (Figs. 4, 6-7, Paragraphs 0047-0052, 0066-0070) multiple synthetic images may be obtained through a homography process which utilizes a common projection plane display the combination of the plurality of second images on a display of the remote operator terminal (Figs. 4, 6-7, Paragraphs 0047-0052, 0061, 0066-0070) the synthetic images may be output to a user through a video screen Regarding claim 5 , Kelly discloses all the limitations of claim 1. Kelly further discloses the following: display the plurality of second images side-by-side on one or more screens of the display (Figs. 4, 6-7, Paragraphs 0047-0053, 0061, 0066-0070) multiple images (real and synthetic) may be displayed to an operator in a side-by-side manner 0052 specifically discloses that more than one synthetic camera/images may be generated and 0061 at least discloses the display of these cameras/images Regarding claim 6 , Kelly discloses all the limitations of claim 1. Kelly further discloses the following: estimating a direction and an amount of movement of the moving body in a period from the first timing to the second timing (Figs. 4, 6-7, Paragraphs 0047-0053, 0061, 0066-0070, 0080-0082) the homography process may be used to generate synthetic images based on first images obtained by a real camera the system may predict the viewing direction of the synthetic camera and generate synthetic images based on predicted locations of the apparatus (which can be a vehicle) calculating a difference between the first point of view and the second point of view based on the direction and the amount of movement of the moving body (Figs. 4, 6-7, Paragraphs 0047-0053, 0061, 0066-0070, 0080-0082) the homography process used to generate the synthetic images are based on movement of the apparatus which includes a given direction (0081 discloses movement along a road which is a direction) the predicted locations from movement are used to determine differences between synthetic images, which includes differing points of view converting the first image viewed from the first point of view into the second image viewed from the second point of view based on the difference between the first point of view and the second point of view (Figs. 4, 6-7, Paragraphs 0047-0053, 0061, 0066-0070, 0080-0082) the synthetic images obtained through homography and are generated based on differences between predicted positions of the apparatus these synthetic images are based off of real camera images obtained through the one or more real cameras through the homography process which is a conversion between real and synthetic images Regarding claim 7 , Kelly discloses all the limitations of claim 1. Kelly further discloses the following: set at least a part of a communication delay time between the moving body and the remote operator terminal as a delay compensation time (Figs. 4, 6-7, Paragraphs 0047-0053, 0061, 0066-0070, 0080-0082) latency may be utilized in generating the synthetic images through homography this latency can be considered the delay compensation time as the system utilizes it when generating the synthetic images set the second timing to be later than the first timing by the delay compensation time (Figs. 4, 6-7, Paragraphs 0047-0053, 0061, 0066-0070, 0080-0082) sequential synthetic images may be generated based on a latency which result in the sequential synthetic images to be at a later time than those images taken by the real camera Regarding claim 8 , the claim limitations are similar to those in claim 1 and are rejected using the same rationale as seen above in claim 1. Regarding claim 9 , the claim limitations are similar to those in claim 1 and are rejected using the same rationale as seen above in claim 1. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claim 2 are rejected under 35 U.S.C. 103 as being obvious over Kelly and in view of Brudnak, US 20190058853 A1, herein referred to as Brudnak. Regarding claim 2 , Kelly discloses a remote operator (Figs. 4, 6-7, Paragraph 0047; a remote operator may be used for controlling the remote apparatus) and multiple cameras (Figs. 4, 6-7, Paragraphs 0047-0052, 0066-0070; multiple cameras may be utilized), but fails to disclose select one of the plurality of cameras as a reference camera, and set the projection plane associated with the reference camera as the common projection plane. However, Brudnak, in an analogous field of endeavor, teaches select one of the plurality of cameras as a reference camera (at least Paragraphs 0046, 0054; a camera may be utilized for setting the reference planes for image projection; utilizing this camera can be considered as selecting a main camera as all projections and references are based on this camera), and set the projection plane associated with the reference camera as the common projection plane (at least Paragraphs 0046, 0054; a projection plane may be associated with the camera which is used as a common plane for all projections). Therefore, from the teaching of Brudnak, it would have been obvious to one of ordinary skill in the art before the effective filing date to have modified, with a reasonable expectation for success, the remote system of Kelly to include select one of the plurality of cameras as a reference camera, and set the projection plane associated with the reference camera as the common projection plane, as taught/suggested by Brudnak. The motivation to do so would be to define the reference plane for the homography process. This can ensure that projections are based on a known baseline reference, which can lead to increased accuracy and efficiency in the system. Claims 3-4 are rejected under 35 U.S.C. 103 as being obvious over Kelly, in view of Brudnak, and further in view of Otani et al., US 20210381197 A1, herein referred to as Otani. Regarding claim 3 , Kelly in view of Brudnak renders obvious all the limitations of claim 2. Kelly further discloses multiple cameras with different field of views (Figs. 4, 6-7, Paragraphs 0047-0052, 0066-0070; multiple cameras may be utilized that have different field of views), but fails to disclose estimate a gaze direction of the remote operator, and dynamically select one corresponding to the gaze direction among the plurality of cameras as the reference camera. However, Otani, in an analogous field of endeavor, teaches estimate a gaze direction of the remote operator (at least Paragraphs 0086-0097; a user’s direction of view may be determined based on their posture), and dynamically select one corresponding to the gaze direction among the plurality of cameras as the reference camera (at least Paragraphs 0086-0097; a camera of a plurality of cameras may be selected based on the user’s estimated gaze direction). Therefore, from the teaching of Otani, it would have been obvious to one of ordinary skill in the art before the effective filing date to have further modified, with a reasonable expectation for success, the remote system of Kelly and Brudnak to include estimate a gaze direction of the remote operator, and dynamically select one corresponding to the gaze direction among the plurality of cameras as the reference camera, as taught/suggested by Otani. The motivation to do so would be to provide relevant view points for a user based on their gaze direction. This can help the user better understand the environment by providing a projected view that is most aligned with their own view. This can lead to increased accuracy in remote vehicle operations as the environment is better presented to the user. Regarding claim 4 , Kelly in view of Brudnak, and further in view of Otani renders obvious all the limitations of claim 3. Kelly further discloses multiple cameras with different field of views (Figs. 4, 6-7, Paragraphs 0047-0052, 0066-0070; multiple cameras may be utilized that have different field of views), but fails to disclose estimate the gaze direction of the remote operator based on a direction of a line of sight of the remote operator or a steering angle of a steering operation performed by the remote operator. However, Otani teaches estimate the gaze direction of the remote operator based on a direction of a line of sight of the remote operator or a steering angle of a steering operation performed by the remote operator (at least Paragraphs 0086-0097; a camera of a plurality of cameras may be selected based on the user’s line of sight which is based on their posture). Therefore, from the teaching of Otani, it would have been obvious to one of ordinary skill in the art before the effective filing date to have further modified, with a reasonable expectation for success, the remote system of Kelly, Brudnak, and Otani to include estimate the gaze direction of the remote operator based on a direction of a line of sight of the remote operator or a steering angle of a steering operation performed by the remote operator, as taught/suggested by Otani. The motivation to do so would be to provide relevant view points for a user based on their gaze direction/line of sight. This can help the user better understand the environment by providing a projected view that is most aligned with their own view. This can lead to increased accuracy in remote vehicle operations as the environment is better presented to the user. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER ALLEN BUKSA whose telephone number is (571)272-5346. The examiner can normally be reached M-F 7:30 AM-4:30 PM. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thomas Worden can be reached at (571) 272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /CHRISTOPHER A BUKSA/Examiner, Art Unit 3658 Application/Control Number: 19/045,174 Page 2 Art Unit: 3658 Application/Control Number: 19/045,174 Page 3 Art Unit: 3658 Application/Control Number: 19/045,174 Page 4 Art Unit: 3658 Application/Control Number: 19/045,174 Page 5 Art Unit: 3658 Application/Control Number: 19/045,174 Page 6 Art Unit: 3658 Application/Control Number: 19/045,174 Page 7 Art Unit: 3658 Application/Control Number: 19/045,174 Page 8 Art Unit: 3658 Application/Control Number: 19/045,174 Page 9 Art Unit: 3658 Application/Control Number: 19/045,174 Page 10 Art Unit: 3658 Application/Control Number: 19/045,174 Page 11 Art Unit: 3658
Prosecution Insights