DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of claims
Claims 1-11 are amended.
No new claim is added.
Claims 1-11 are pending.
Response to arguments
With respect to Applicant’s remarks filed on 02/04/2026; Applicant's “Amendments and Remarks” have been fully considered. Applicant’s remarks will be addressed in sequential order as they were presented. Applicant has amended the independent claim and these amendments have changed the scope of the original application and the Office has supplied new grounds for rejection attached below in the FINAL office action and therefore the prior arguments are considered moot.
Applicant remarks:
Amended claims do not recite a mental process or mathematical concepts.
Yoshizawa is not shown to describe acquiring a second three-dimensional trajectory from a slave apparatus. Further, Veelaret and Chen do not remedy the above noted deficiencies of Yoshizawa.
Office Response:
Amended claims overcome 35 U.S.C. 101 rejection. Specially, “generating a synchronized map by synchronization of the first map with the second map” cannot perform in human mind.
Please see new mapping above, specifically the mapping for the independent claims.
Applicant further argues that the other independent claims which recite similar features are allowable and the dependent claims are also allowable since they depend on allowable subject and the Office respectfully disagrees. It is the Office's stance that all of the claimed subject matter has been properly rejected; therefore, the Office's respectfully disagrees with applicant’s arguments.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 10-11 are rejected under 35 U.S.C. 103 as being unpatented over EP3855400A1 to Jiang et al. (herein after “Jiang”) in view of CN115035031B to San et al. (herein after “San”).
Regarding claim 1, Jiang teaches A first information processing
apparatus, comprising:
a first camera configured to capture an image of a slave apparatus. wherein the slave apparatus is a second information processing apparatus that functions as a slave; (see Jiang first terminal as master and second terminal as slave)
and (See Jiang para[0008] collecting, by a first terminal, a first image by using a camera, where the first image includes an image of any object in any natural environment (real world);)
a central processing unit (CPU) configured to: (See Jiang para [0137] The processor 403 may be one or more central processing units (Central Processing Unit, CPU). When the processor 403 is one CPU, the CPU may be a single-core CPU or may be a multi-core CPU.)
generate a first map;(See Jiang para[0019] where the map data is generated based on first point cloud data of any object,)
estimate a first chronological self-position of the first camera in the first map ; (See Jiang para [0018] Based on the first aspect, in a possible embodiment, before the generating, by the first terminal, map data based on the first point cloud data, the pose data, and the two-dimensional feature image, the method further includes: obtaining, by the first terminal, positioning data,)
generate a first three-dimensional trajectory of the first camera based on the estimated first chronological self-position in the first map; (See Jiang para[0019] the first point cloud data indicates three-dimensional coordinate values of a surface point of the object in the first camera coordinate system of the first terminal;)
acquire a second three- dimensional trajectory of a second camera from the slave apparatus via a network (See Jiang para[0026] The second terminal obtains second point cloud data of the object based on the second image, where the second point cloud data indicates three-dimensional coordinate values of a surface point of the object in the second camera coordinate system wherein
the slave apparatus includes the second camera, and the slave apparatus:
(See Jiang para[0019] obtaining, by the second terminal, pose data of the second terminal in a second camera coordinate system of the second terminal;)
estimates a second chronological self-position of the second camera in the second map and (See Jiang para[0019] obtaining, by the second terminal, pose data of the second terminal in a second camera coordinate system of the second terminal; obtaining, by the second terminal, a two-dimensional feature image of the second terminal;)
generates the second three-dimensional trajectory based on the estimated second chronological self-position in the second map: (See Jiang para[0026] The second terminal obtains second point cloud data of the object based on the second image, where the second point cloud data indicates three-dimensional coordinate values of a surface point of the object in the second camera coordinate system.)
generate an observed two-dimensional trajectory, wherein the observed two-dimensional trajectory indicates a chronological position of the slave apparatus in the captured image of the slave apparatus; (See Jiang para[0019] a two-dimensional feature image of the second terminal; para[0031] receiving, by the server, pose data of a second terminal in a second camera coordinate system of the second terminal and a two-dimensional feature image of the second terminal; performing, by the server, a feature matching operation in the map data by using the two-dimensional feature image of the second terminal, to obtain pose data of the second terminal in the first camera coordinate system)
calculate a transformation parameter based on the first three-dimensional trajectory, the second three-dimensional trajectory, and the observed two-dimensional trajectory (See Jiang para[0045] obtain coordinate system transformation data between the first camera coordinate system and the second camera coordinate system based on the pose data of the second terminal in the first camera coordinate system and the pose data of the second terminal in the second camera coordinate system; and a sending module, configured to send the coordinate system transformation data to the second terminal.) ; and
generate a synchronized map by synchronization of the first map with the second map, wherein the synchronization of the first map with the second map is based on the transformation parameter. (See Jiang para[0011] the first terminal can extract an image feature by using output data of a local spatial positioning engine, generate the map data, and send the map data to the second terminal or the server, so that the second terminal implements coordinate system synchronization between the two terminals based on the map data)
However, Jiang does not expressly disclose and otherwise teach generating a second map based on second point cloud data. Nevertheless, San same field of endeavor teaches generates a second map: (see San para[0108] translation matrix (T) between the point cloud data collected by line laser camera L2 and the point cloud data collected by line laser camera L1. R and T are the calibration results of line laser cameras L1 and L2; it is possible to generate map using point cloud data collected from line laser camera L2).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention with a reasonable expectation of success to combine Jiang’s data processing method with San’s second point cloud data in order to generate second map to reliable communication connections and easy assembly/disassembly (see para[0002]).
Regarding claim 2, Jiang and San remain applied as claim 1. Jiang teaches wherein CPU is further configured to: estimate a two-dimensional trajectory of the slave apparatus based on a candidate value of the transformation parameter See Jiang para[0096] output by the spatial positioning engine, project the second point cloud data onto the second image by using the pose, to obtain the 2D image coordinates,), the first three-dimensional trajectory(See Jiang para[0019]), and the second three-dimensional trajectory (See Jiang para[0026]);
optimize the candidate value of the transformation parameter to coincide the estimated two-dimensional trajectory of the slave apparatus(See Jiang para[0096] output by the spatial positioning engine, project the second point cloud data onto the second image by using the pose, to obtain the 2D image coordinates,) with the observed two-dimensional trajectory; and calculate the transformation parameter based on the optimization of the candidate value of the transformation parameter. (See Jiang para[0045] obtain coordinate system transformation data between the first camera coordinate system and the second camera coordinate system based on the pose data of the second terminal in the first camera coordinate system and the pose data of the second terminal in the second camera coordinate system; and a sending module, configured to send the coordinate system transformation data to the second terminal.).
Regarding claim 3, Jiang and San remain applied as claim 1. Jiang teaches wherein CPU is further configured to:
(See Jiang para [0137] The processor 403 may be one or more central processing units (Central Processing Unit, CPU). When the processor 403 is one CPU, the CPU may be a single-core CPU or may be a multi-core CPU.)
calculate the chronological position of the slave apparatus in the first map based on the first three-dimensional trajectory and the second three- dimensional trajectory (See Jiang claim 7 obtaining, by the second terminal, the coordinate system transformation data between the first camera coordinate system and the second camera coordinate system based on the pose data of the second terminal in the first camera coordinate system and the pose data of the second terminal in the second camera coordinate system); and
optimize the candidate value of the transformation parameter based on the calculated chronological position of the slave apparatus in the first map. (See Jiang para[0010] the server is configured to obtain coordinate system transformation data between the first camera coordinate system of the first terminal and a second camera coordinate system of the second terminal based on the map data, and send the coordinate system transformation data to the second terminal, so that the second terminal synchronizes content in the virtual scenario based on the coordinate system transformation data.)
Regarding claim 4, Jiang and San remain applied as claim 1. Jiang teaches wherein the CPU is further configured to: (See Jiang para [0137] The processor 403 may be one or more central processing units (Central Processing Unit, CPU). When the processor 403 is one CPU, the CPU may be a single-core CPU or may be a multi-core CPU.) and:
generate the synchronized map based on the transformation of the first map. (See Jiang para[0011] the first terminal can extract an image feature by using output data of a local spatial positioning engine, generate the map data, and send the map data to the second terminal or the server, so that the second terminal implements coordinate system synchronization between the two terminals based on the map data)
However, Jiang does not expressly disclose and otherwise teach transform the first map based on the transformation parameter. Nevertheless, San same field of endeavor teaches transform the first map based on the transformation parameter (See San Converting the point cloud data acquired by each slave camera into the coordinate system of the master camera based on a conversion relation between the slave cameras and the coordinate system of the master camera;).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention with a reasonable expectation of success to combine Jiang’s data processing method with San’s second point cloud data in order to generate second map to reliable communication connections and easy assembly/disassembly (see para[0002]).
Regarding claim 5, Jiang and San remain applied as claim 1. Jiang teaches wherein the CPU is further configured to:(See Jiang para [0137] The processor 403 may be one or more central processing units (Central Processing Unit, CPU). When the processor 403 is one CPU, the CPU may be a single-core CPU or may be a multi-core CPU.)
transmit the transformation parameter to the slave apparatus, (See Jiang para[0010] When the first terminal sends the map data to the server, the server is configured to obtain coordinate system transformation data between the first camera coordinate system, para[0008] and sending, by the first terminal, the map data to a second terminal or a server)
generate the synchronized map based on the transformation of the second map. (See Jiang para[0032] The server further obtains coordinate system synchronization data based on the key frame and the map data, and delivers the coordinate system synchronization data to the second terminal, to implement coordinate system synchronization between the terminals.)
However, Jiang does not expressly disclose and otherwise teach transform the second map based on the transformation parameter. Nevertheless, San same field of endeavor teaches transform the second map based on the transformation (See San claim 1 Converting the point cloud data acquired by each slave camera into the coordinate system of the master camera based on a conversion relation between the slave cameras and the coordinate system of the master camera;).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention with a reasonable expectation of success to combine Jiang’s data processing method with San’s second point cloud data in order to generate second map to reliable communication connections and easy assembly/disassembly (see para[0002]).
Regarding claim 6, Jiang and San remain applied as claim 1. Jiang teaches wherein the CPU is further configured to: generate an augmented reality (ARI object based on the synchronized map (See Jiang para[0008] According to a first aspect, an embodiment of the present invention provides a data processing method used in a virtual scenario. The virtual scenario is one of a virtual reality VR scenario, an augmented reality AR scenario); and
display the generated AR object on a display apparatus. (See Jiang claim 5 wherein the virtual scenario is one of a virtual reality VR scenario, an augmented reality AR scenario, and a mixed reality MR scenario;para[0129]).
Regarding claim 7, Jiang and San remain applied as claim 1. Jiang teaches wherein the first information processing apparatus functions as a master (See Jiang first terminal consider as master which control second terminal)
: and the slave apparatus generates the AR object of common AR content, wherein the common AR (See Jiang para[0008] According to a first aspect, an embodiment of the present invention provides a data processing method used in a virtual scenario. The virtual scenario is one of a virtual reality VR scenario, an augmented reality AR scenario) content is common to the first information processing apparatus and the slave apparatus; and
displays the generated AR object of the common AR content. (See Jiang claim 5 wherein the virtual scenario is one of a virtual reality VR scenario, an augmented reality AR scenario, and a mixed reality MR scenario;para[0129]).
Regarding claim 9, Jiang and San remain applied as claim 1. Jiang teaches wherein the first chronological self-position includes a position and an attitude of the first camera. (See Jiang para[00005] a pose of the terminal relative to the reference coordinate system may be calculated by using a pose output by a spatial positioning engine of the mobile terminal, and then a relative pose between terminals is calculated,para[0008] by the first terminal, pose data of the first terminal in the first camera coordinate system based on the first image).
Regarding claim 10, Jiang teaches An information processing method, comprising: controlling a first camera to capture an image of a slave apparatus, wherein the slave apparatus is an information processing apparatus that functions as a slave; (See Jiang a camera controller 319 in para[0129])
generating a first map;(See Jiang para[0019] where the map data is generated based on first point cloud data of any object,)
estimating a first chronological self-position of the first camera in the first map; (See Jiang para [0018] Based on the first aspect, in a possible embodiment, before the generating, by the first terminal, map data based on the first point cloud data, the pose data, and the two-dimensional feature image, the method further includes: obtaining, by the first terminal, positioning data,)
generating a first three-dimensional trajectory of the first camera based on the estimated first chronological self-position in the first map; (See Jiang para[0019] the first point cloud data indicates three-dimensional coordinate values of a surface point of the object in the first camera coordinate system of the first terminal;)
acquiring a second three-dimensional trajectory of a second camera from the slave apparatus via a network, wherein (See Jiang para[0026] The second terminal obtains second point cloud data of the object based on the second image, where the second point cloud data indicates three-dimensional coordinate values of a surface point of the object in the second camera coordinate system.)
the slave apparatus includes the second camera(See Jiang para[0019] obtaining, by the second terminal, pose data of the second terminal in a second camera coordinate system of the second terminal;), and
to generate the second three-dimensional traiectory, the slave apparatus: (See Jiang para[0026] The second terminal obtains second point cloud data of the object based on the second image, where the second point cloud data indicates three-dimensional coordinate values of a surface point of the object in the second camera coordinate system.)
estimates a second chronological self-position of the second camera in the second map(See Jiang para[0019] obtaining, by the second terminal, pose data of the second terminal in a second camera coordinate system of the second terminal; obtaining, by the second terminal, a two-dimensional feature image of the second terminal;);and
generates the second three-dimensional traiectory based on the estimated second chronological self-position in the second map: (See Jiang para[0026] The second terminal obtains second point cloud data of the object based on the second image, where the second point cloud data indicates three-dimensional coordinate values of a surface point of the object in the second camera coordinate system.)
generating an observed two-dimensional trajectory, wherein the observed two- dimensional trajectory indicates a chronological position of the slave apparatus in the captured image of the slave apparatus; (See Jiang para[0019] a two-dimensional feature image of the second terminal; para[0031] receiving, by the server, pose data of a second terminal in a second camera coordinate system of the second terminal and a two-dimensional feature image of the second terminal; performing, by the server, a feature matching operation in the map data by using the two-dimensional feature image of the second terminal, to obtain pose data of the second terminal in the first camera coordinate system)
calculating a transformation parameter based on the first three-dimensional trajectory, the second three-dimensional trajectory, and the observed two-dimensional trajectory(See Jiang para[0045] obtain coordinate system transformation data between the first camera coordinate system and the second camera coordinate system based on the pose data of the second terminal in the first camera coordinate system and the pose data of the second terminal in the second camera coordinate system; and a sending module, configured to send the coordinate system transformation data to the second terminal.) ;_and
generating a synchronized map by synchronization of the first map with the second map, wherein the synchronization of the first map with the second map is based on the transformation parameter. (See Jiang para[0011] the first terminal can extract an image feature by using output data of a local spatial positioning engine, generate the map data, and send the map data to the second terminal or the server, so that the second terminal implements coordinate system synchronization between the two terminals based on the map data).
However, Jiang does not expressly disclose and otherwise teach generates a second map. Nevertheless, San same field of endeavor teaches generates a second map (see San para[0108] translation matrix (T) between the point cloud data collected by line laser camera L2 and the point cloud data collected by line laser camera L1. R and T are the calibration results of line laser cameras L1 and L2; it is possible to generate map using point cloud data collected from line laser camera L2).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention with a reasonable expectation of success to combine Jiang’s data processing method with San’s second point cloud data in order to generate second map to reliable communication connections and easy assembly/disassembly (see para[0002]).
Regarding claim 11, Jiang teaches A non-transitory computer-readable medium having stored thereon, computer-executable instructions which, when executed by a central processing unit (CPU) (See Jiang para [0137] ) of a first information processing apparatus, cause the first information processing apparatus to execute operations, the operations comprising (see Jiang at least para[0050] When the program code is executed by a computing device, the computing device is configured to perform the method in the first aspect)
controlling a first camera to capture an image of a slave apparatus, wherein the slave apparatus is a second information processing apparatus that functions as a slave; (See Jiang a camera controller 319 in para[0129])
generates generating a first map; ;(See Jiang para[0019] where the map data is generated based on first point cloud data of any object,)
estimating a first chronological self-position of the first camera in the first map; (See Jiang para [0018] Based on the first aspect, in a possible embodiment, before the generating, by the first terminal, map data based on the first point cloud data, the pose data, and the two-dimensional feature image, the method further includes: obtaining, by the first terminal, positioning data,)
generating a first three-dimensional trajectory of the first camera based on the estimated first chronological self-position in the first map; (See Jiang para[0019] the first point cloud data indicates three-dimensional coordinate values of a surface point of the object in the first camera coordinate system of the first terminal;)
acquiring a second three- dimensional trajectory of a second camera from the slave apparatus via a network(See Jiang para[0026] The second terminal obtains second point cloud data of the object based on the second image, where the second point cloud data indicates three-dimensional coordinate values of a surface point of the object in the second camera coordinate system.), wherein
the slave apparatus includes the second camera(See Jiang para[0019] obtaining, by the second terminal, pose data of the second terminal in a second camera coordinate system of the second terminal;), and
to generate the second three-dimensional traiectory, the slave apparatus: (See Jiang para[0026] The second terminal obtains second point cloud data of the object based on the second image, where the second point cloud data indicates three-dimensional coordinate values of a surface point of the object in the second camera coordinate system, it is possible to generate trajectory since Jiang teaches in para[0003] the terminal performs self-positioning based on a location estimation and a map, and constructs an incremental map based on the self-positioning for subsequent positioning)
estimates a second chronological self-position of the second camera in the second map(See Jiang para[0019] obtaining, by the second terminal, pose data of the second terminal in a second camera coordinate system of the second terminal; obtaining, by the second terminal, a two-dimensional feature image of the second terminal;) and
generates the second three-dimensional traiectory based on the estimated second chronological self-position in the second map; (See Jiang para[0026] The second terminal obtains second point cloud data of the object based on the second image, where the second point cloud data indicates three-dimensional coordinate values of a surface point of the object in the second camera coordinate system.)
generating an observed two-dimensional trajectory, wherein the observed two-dimensional trajectory indicates a chronological position of the slave apparatus in the captured image obtained by a camera imaging of the slave apparatus; (See Jiang para[0019] a two-dimensional feature image of the second terminal; para[0031] receiving, by the server, pose data of a second terminal in a second camera coordinate system of the second terminal and a two-dimensional feature image of the second terminal; performing, by the server, a feature matching operation in the map data by using the two-dimensional feature image of the second terminal, to obtain pose data of the second terminal in the first camera coordinate system)
calculating a transformation parameter based on the first three- dimensional trajectory, the second three-dimensional trajectory, and the observed two- dimensional trajectory(See Jiang para[0045] obtain coordinate system transformation data between the first camera coordinate system and the second camera coordinate system based on the pose data of the second terminal in the first camera coordinate system and the pose data of the second terminal in the second camera coordinate system; and a sending module, configured to send the coordinate system transformation data to the second terminal.); and
generating a synchronized map by synchronization of the first map with the second map, wherein the synchronization of the first map with the second map is based on the transformation parameter. (See Jiang para[0011] the first terminal can extract an image feature by using output data of a local spatial positioning engine, generate the map data, and send the map data to the second terminal or the server, so that the second terminal implements coordinate system synchronization between the two terminals based on the map data)
However, Jiang does not expressly disclose and otherwise teach generates a second map. Nevertheless, San same field of endeavor teaches generates a second map (see San para[0108] translation matrix (T) between the point cloud data collected by line laser camera L2 and the point cloud data collected by line laser camera L1. R and T are the calibration results of line laser cameras L1 and L2; it is possible to generate map using point cloud data collected from line laser camera L2).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention with a reasonable expectation of success to combine Jiang’s data processing method with San’s second point cloud data in order to generate second map to reliable communication connections and easy assembly/disassembly (see para[0002]).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatented over Jiang in view of San and US 20200311040 A1 to Wada et al. (herein after “Wada”).
Regarding claim 8, Jiang and San remain applied as claim 1. Jiang teaches wherein the CPU (See Jiang para [0137] The processor 403 may be one or more central processing units (Central Processing Unit, CPU). When the processor 403 is one CPU, the CPU may be a single-core CPU or may be a multi-core CPU.). However, Jiang does not teach is further configured to transmit the second three-dimensional trajectory a third information processing apparatus, and the third information processing apparatus functions as a master based on the slave apparatus that functions as the slave. Nevertheless, Wada same field of endeavor teaches to transmit the second three-dimensional trajectory a third information processing apparatus, and the third information processing apparatus functions as a master based on the slave apparatus that functions as the slave. (see Wada para[0032] FIG. 20 depicts a block transfer from slave to master. ; and figure 20, para[0142]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention with a reasonable expectation of success to combine Jiang’s data processing method with Wada’s transmission from slave to master transaction in order to allow to dynamically register and render an application programmer's interface in a cloud services infrastructures (see para[0002]).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NAZIA AFRIN whose telephone number is (703)756-1175. The examiner can normally be reached Monday-Friday 7:30-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Scott A Browne can be reached at 5712700151. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NAZIA AFRIN/ Examiner, Art Unit 3666
/SCOTT A BROWNE/ Supervisory Patent Examiner, Art Unit 3666
Prosecution Insights