DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record on file.
Preliminary Amendment
The preliminary amendment filed on July 28th, 2023 has been acknowledged and entered.
Information Disclosure Statement(s)
The Information disclosure statements (IDS) filed on July 28th, 2023, March 08th, 2024, August 16th, 2024 and September 14th, 2025 have been acknowledged and considered by the examiner.
Response to Remarks
The Office Action has been made issued in response to Response to Election/Restriction Requirement filed January 15th, 2024. Claims 1-7 and 12-20 are pending.
The amendment to the claims filed on March 30th, 2026 along with the filed Response to Election/Restriction has been acknowledged and entered.
Claims 1 and 12 have been amended, claims 10-11 (of nonelected group) have been canceled.
Election/Restrictions
Claims 8-11 of Species B are withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to a nonelected Species, respectively, there being no allowable generic or linking claim. Election was made without traverse in the reply filed on March 30th, 2026. Accordingly, Claims 1-7, 12-20 of Species A are being pending based on the elected Species A by the applicants. Applicants are respectfully reminded that the restriction is made based on the original presentation of the claims, such as for the Applicants’ claims filed on July 28th, 2023, previously made the restriction mailed on January 29th, 2026 stating clearly the distinction between the claim groupings. The current Office Action lays out the rejections for the Applicants’ selected claims, as stated in the response section above.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 13 is rejected under 35 U.S.C. 101
The claimed invention is directed to non-statutory subject matter. The claims do not fall within at least one of the four categories of patent eligible subject matter because the claims recite a program product with instructions stored in a computer-readable storage medium, to be executed by a processor to cause an electronic device (not part of the program product, not explicitly recited as part of the scope of the claims) with a computer-readable medium, however, the instant specification’s paragraph 198 provides that the computer-readable medium can be of a carrier signal or in some jurisdiction (not in all instances) it cannot be a carrier signal (by BRI, can be understood as in other jurisdiction, it can be carrier signal); therefore, the examiner interprets the claims, based on BRI, to be a carrier or telecommunication signal which is signal per se, rejected 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 5-6, 12-13 and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Adam Rowell et. al. (“US 10,602,126 B2” hereinafter as “Rowell”) in view of William L. Gaddy et. al. (“US 8,988,502” hereinafter as “Gaddy”) and Chen Wu et. al. (“US 2014/0118494 A1” hereinafter as “Wu”).
Regarding claim 1, Rowell discloses a method for processing a three-dimensional video, comprising (Abstract discloses processing 3D data video feed):
acquiring, from a cloud server (Col. 11, lines 23-30, discloses “the master streaming server transmits image and video files…the streaming serve includes a TCP connection for sending a TCP stream to a second TCP connection included in the imaging system” indicating a cloud server as claimed since, TCP connection transmits over a network), depth video streams (Column 10, lines 45-60, discloses “post processing…may be processed by a master streaming service…produces image and video files…the master streaming client may also encode image and/or video files into image and/or video streams;” moreover, Col. 10, lines 61-67 and Col. 11, lines 1-13, disclose “the master streaming client may also encode depth information…and/or video stream” indicating acquiring depth video streams being associated with each other) from perspectives of at least two cameras of a same scene (Col. 10, lines 1-10, discloses “having multiple cameras….used to capture the visual aspects of a scene” indicating perspectives [visual aspects] of at least two cameras of a same scene), wherein each of the depth video streams comprises a Red-Green-Blue (RGB) stream (Column 10, lines 10-26, discloses “example sensors that may be incorporated in the digital camera device include….visible light sensors (e.g., RGB cameras);” moreover, Column 42, lines 42-53, discloses “the auto re-calibration subsystem can work on any digital camera device….to ensure digital re-calibration process can run in parallel with sensor control,….,depth data processing” indicating the digital camera includes RGB data and depth data being processed in parallel [together], therefore, the video streams as discussed comprises RGB stream) and a depth information stream (as discussed previously, Column 42, lines 42-53, indicating the digital camera includes RGB data and depth data being processed in parallel [together], therefore, the video streams as discussed comprises depth information stream), the RGB stream is to the cloud server (Col. 11, lines 23-30, as discussed previously, disclosing the streaming server to transmit the video file to a second TCP connection [analogous to the cloud server];
registering, according to preset registration information, the depth video streams from the perspectives of the at least two cameras (Col. 29, lines 44-52, discloses “align an image captured by one camera module to another image captured by another camera module;” furthermore, Col. 40, lines 53-67, discloses “over time, the position of one or more camera modules may shift…calibration process rectification by offsetting the alignment of images and videos generated by left and right camera modules” indicating a registering between video streams to align the views of the cameras [perspectives of the cameras] according to offsetting [preset registration information]); and
performing, according to the registered depth video streams from the perspectives of the at least two cameras, three-dimensional reconstruction to obtain a 3D video (Col. 3, lines 41-60, discloses “the digital camera device cannot function properly without effective calibration and rectification techniques….delivering output of every digital camera device….including 3D content generation, scene reconstruction” indicating using of the result of the camera calibration [registered depth video streams as discussed previously] to perform scene reconstruction [3D reconstruction of the video data]).
However, Rowell does not explicitly disclose the RGB stream is to the cloud server through a respective one of RGB channels, and the depth information stream is evenly distributed to the RGB channels and sent to the cloud server through the RGB channels.
In the same field of depth-image-based processing (Abstract, Gaddy) Gaddy discloses the RGB stream is to the cloud server through a respective one of RGB channels (Col. 2, lines 52-65, discloses “method for transmitting 3D imagery, comprising…depth map and color data” including 3D video transmission of the data [disclosed in Col. 1, lines 28-30]), and the depth information stream distributed to the RGB channels and sent to the cloud server through the RGB channels (Col. 2, lines -52-65, discloses “a depth map of a single channel of image data, where each pixel of the data represents the range or distance of each pixel of a corresponding color image” indicating the correspondence between one pixel of the depth data corresponds to one pixel of the color image through channel of image data [RGB channels of the color data]).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell’s method of acquiring depth video streams from perspective of at least two cameras of a same scene, wherein each of the depth video streams comprises a Red-Green-Blue stream and depth information stream;
Wherein Rowell’s acquiring of the RGB stream which is to a cloud server through a respective one of RGB channels can be modified to have the depth information stream is distributed to the RGB channels and sent to the cloud server through the RGB channels as taught by Gaddy as discussed in the mapping above.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to have a method that can transmit 3D imagery along with depth data in the manner such as discussed in a low or no incremental bandwidth cost and perform high quality reconstruction of image data (Gaddy’s Col. 2, lines 41-48).
However, Rowell in view of Gaddy does not explicitly disclose the depth information stream is evenly distributed to the RGB channels and sent to the cloud server through the RGB channels.
In the same field of depth and color data of image processing (Abstract, Wu), Wu discloses the depth information stream is evenly distributed to the RGB channels and sent to the cloud server through the RGB channels (Par. [0040] discloses “the depth map module computes…a uniform histogram…The uniform histogram, representing an equal distribution of all colors in each channel…in the Y, Cb, and Cr channels…the depth map module determines the color depth map weight…of the histograms” indicating that RGB-D data can be represented through uniform histogram with the depth data is evenly distributed [uniform distribution] of all colors in each channel).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell’s method of acquiring depth video streams from perspective of at least two cameras of a same scene, wherein each of the depth video streams comprises a Red-Green-Blue stream and depth information stream; wherein Rowell’s acquiring of the RGB stream which is to a cloud server through a respective one of RGB channels can be modified to have the depth information stream is distributed to the RGB channels and sent to the cloud server through the RGB channels as taught by Gaddy and the depth information stream is evenly distributed to the RGB channels as taught by Wu as discussed in the mapping above.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to use a way of representing RGB-D data through distribution of colors in the image and corresponding depth data, so that it’s beneficial to increase the color depth map weight when the color distribution is wide, so that the RGB-D data can be managed effectively during related processing (Wu’s Par. [0038]).
Regarding claim 5, Rowell in view of Gaddy and Wu, wherein Rowel discloses the method of claim 1, wherein after obtaining the 3D video, the method further comprises: acquiring perspective information (Col. 10, lines 1-10, discloses “having multiple cameras….used to capture the visual aspects of a scene” indicating perspectives [visual aspects] of at least two cameras of a same scene), and determining, according to the perspective information, a target image (Column 10, lines 10-26, discloses “example sensors that may be incorporated in the digital camera device include….visible light sensors (e.g., RGB cameras);” moreover, Column 42, lines 42-53, discloses “the auto re-calibration subsystem can work on any digital camera device….to ensure digital re-calibration process can run in parallel with sensor control,….,depth data processing” indicating the digital camera includes RGB data and depth data being processed in parallel [together], therefore, the video streams as discussed comprises RGB stream, indicating a target image); and sending the target image to a playback device for playback (FIG. 4 illustrates, in step 490, a repeating of a procedure for both left and right displays, indicating a playback on a display device).
Regarding claim 6, Rowell in view of Gaddy and Wu, wherein Rowel discloses the method of claim 5, wherein determining, according to the perspective information, the target image (Column 10, lines 10-26, discloses “example sensors that may be incorporated in the digital camera device include….visible light sensors (e.g., RGB cameras);” moreover, Column 42, lines 42-53, discloses “the auto re-calibration subsystem can work on any digital camera device….to ensure digital re-calibration process can run in parallel with sensor control,….,depth data processing” indicating the digital camera includes RGB data and depth data being processed in parallel [together], therefore, the video streams as discussed comprises RGB stream, indicating a target image), comprises: configuring, according to the perspective information, a virtual camera (FIG. 4 illustrates, at step 490, a procedure on a VR headset [virtual camera], as the result of the previous steps 410-480 including the determination as discussed); and determining an image photographed by the virtual camera as a target image (Col. 10, lines 1-10, discloses “having multiple cameras….used to capture the visual aspects of a scene” indicating perspectives [visual aspects] of at least two cameras of a same scene, the camera of a VR headset would be a virtual camera and the image captured by the system would be a target image).
Regarding claim 12, Rowell discloses an electronic device, comprising: at least one processing apparatus (Fig. 1 illustrated, in component 106, a processor to perform the invention’s process); and a storage apparatus configured to store at least one program (Fig. 1 illustrates, in component 105, a memory storing the invention’s process to be executed by the processor); wherein the at least one program, when executed by the at least one processing apparatus, causes the at least one processing apparatus (Fig. 1 illustrates the system of the invention including a processor executing instructions stored in the memory as discussed) to perform:
acquiring, from a cloud server (Col. 11, lines 23-30, discloses “the master streaming server transmits image and video files…the streaming serve includes a TCP connection for sending s TCP stream to a second TCP connection included in the imaging system” indicating a cloud server as claimed since, TCP connection transmits over a network), depth video streams (Column 10, lines 45-60, discloses “post processing…may be processed by a master streaming service…produces image and video files…the master streaming client may also encode image and/or video files into image and/or video streams;” moreover, Col. 10, lines 61-67 and Col. 11, lines 1-13, disclose “the master streaming client may also encode depth information…and/or video stream” indicating acquiring depth video streams being associated with each other) from perspectives of at least two cameras of a same scene (Col. 10, lines 1-10, discloses “having multiple cameras….used to capture the visual aspects of a scene” indicating perspectives [visual aspects] of at least two cameras of a same scene), wherein each of the depth video streams comprises a Red-Green-Blue (RGB) stream (Column 10, lines 10-26, discloses “example sensors that may be incorporated in the digital camera device include….visible light sensors (e.g., RGB cameras);” moreover, Column 42, lines 42-53, discloses “the auto re-calibration subsystem can work on any digital camera device….to ensure digital re-calibration process can run in parallel with sensor control,….,depth data processing” indicating the digital camera includes RGB data and depth data being processed in parallel [together], therefore, the video streams as discussed comprises RGB stream) and a depth information stream (as discussed previously, Column 42, lines 42-53, indicating the digital camera includes RGB data and depth data being processed in parallel [together], therefore, the video streams as discussed comprises depth information stream), the RGB stream is to the cloud server (Col. 11, lines 23-30, as discussed previously, disclosing the streaming server to transmit the video file to a second TCP connection [analogous to the cloud server];
registering, according to preset registration information, the depth video streams from the perspectives of the at least two cameras (Col. 29, lines 44-52, discloses “align an image captured by one camera module to another image captured by another camera module;” furthermore, Col. 40, lines 53-67, discloses “over time, the position of one or more camera modules may shift…calibration process rectification by offsetting the alignment of images and videos generated by left and right camera modules” indicating a registering between video streams to align the views of the cameras [perspectives of the cameras] according to offsetting [preset registration information]); and
performing, according to the registered depth video streams from the perspectives of the at least two cameras, three-dimensional reconstruction to obtain a 3D video (Col. 3, lines 41-60, discloses “the digital camera device cannot function properly without effective calibration and rectification techniques….delivering output of every digital camera device….including 3D content generation, scene reconstruction” indicating using of the result of the camera calibration [registered depth video streams as discussed previously] to perform scene reconstruction [3D reconstruction of the video data]);
or (“or” indicates a selection, only one option is the instant scope of the claim, the examiner only selects the previous set of steps to be the instant scope of the claim for mapping),
wherein the at least one program, when executed by the at least one processing apparatus, causes the at least one processing apparatus to perform: acquiring depth video streams from perspectives of at least two cameras of a same scene, wherein each of the depth video streams comprises a Red-Green-Blue (RGB) stream and a depth information stream; and for each of the depth video streams from a respective one of the perspectives of the at least two cameras, sending the RGB stream to a cloud server through a respective one of RGB channels; evenly distributing the depth information stream to the RGB channels, and sending the depth information stream to the cloud server through the RGB channels (the steps are not selected according to the “or” indicator of the claim).
However, Rowell does not explicitly disclose the RGB stream is to the cloud server through a respective one of RGB channels, and the depth information stream is evenly distributed to the RGB channels and sent to the cloud server through the RGB channels.
In the same field of depth-image-based processing (Abstract, Gaddy) Gaddy discloses the RGB stream is to the cloud server through a respective one of RGB channels (Col. 2, lines 52-65, discloses “method for transmitting 3D imagery, comprising…depth map and color data” including 3D video transmission of the data [disclosed in Col. 1, lines 28-30]), and the depth information stream distributed to the RGB channels and sent to the cloud server through the RGB channels (Col. 2, lines -52-65, discloses “a depth map of a single channel of image data, where each pixel of the data represents the range or distance of each pixel of a corresponding color image” indicating the correspondence between one pixel of the depth data corresponds to one pixel of the color image through channel of image data [RGB channels of the color data]).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell’s method of acquiring depth video streams from perspective of at least two cameras of a same scene, wherein each of the depth video streams comprises a Red-Green-Blue stream and depth information stream;
Wherein Rowell’s acquiring of the RGB stream which is to a cloud server through a respective one of RGB channels can be modified to have the depth information stream is distributed to the RGB channels and sent to the cloud server through the RGB channels as taught by Gaddy as discussed in the mapping above.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to have a method that can transmit 3D imagery along with depth data in the manner such as discussed in a low or no incremental bandwidth cost and perform high quality reconstruction of image data (Gaddy’s Col. 2, lines 41-48).
However, Rowell in view of Gaddy does not explicitly disclose the depth information stream is evenly distributed to the RGB channels and sent to the cloud server through the RGB channels.
In the same field of depth and color data of image processing (Abstract, Wu), Wu discloses the depth information stream is evenly distributed to the RGB channels and sent to the cloud server through the RGB channels (Par. [0040] discloses “the depth map module computes…a uniform histogram…The uniform histogram, representing an equal distribution of all colors in each channel…in the Y, Cb, and Cr channels…the depth map module determines the color depth map weight…of the histograms” indicating that RGB-D data can be represented through uniform histogram with the depth data is evenly distributed [uniform distribution] of all colors in each channel).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell’s method of acquiring depth video streams from perspective of at least two cameras of a same scene, wherein each of the depth video streams comprises a Red-Green-Blue stream and depth information stream; wherein Rowell’s acquiring of the RGB stream which is to a cloud server through a respective one of RGB channels can be modified to have the depth information stream is distributed to the RGB channels and sent to the cloud server through the RGB channels as taught by Gaddy and the depth information stream is evenly distributed to the RGB channels as taught by Wu as discussed in the mapping above.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to use a way of representing RGB-D data through distribution of colors in the image and corresponding depth data, so that it’s beneficial to increase the color depth map weight when the color distribution is wide, so that the RGB-D data can be managed effectively during related processing (Wu’s Par. [0038]).
Regarding claim 13, Rowell in view of Gaddy and Wu, wherein Rowel discloses the method of claim 1, a computer-readable storage medium storing a computer program that when executed by a processing apparatus, performs the method for processing a three-dimensional video of claim 1 (as illustrated in FIG. 1, the system of the invention including memory can be of RAM or ROM to store program executed by a processor of the invention processing).
Regarding claim 17, Rowell in view of Gaddy and Wu, wherein Rowel discloses the electronic device of claim 12, wherein after obtaining the 3D video, the method further comprises: acquiring perspective information (Col. 10, lines 1-10, discloses “having multiple cameras….used to capture the visual aspects of a scene” indicating perspectives [visual aspects] of at least two cameras of a same scene), and determining, according to the perspective information, a target image (Column 10, lines 10-26, discloses “example sensors that may be incorporated in the digital camera device include….visible light sensors (e.g., RGB cameras);” moreover, Column 42, lines 42-53, discloses “the auto re-calibration subsystem can work on any digital camera device….to ensure digital re-calibration process can run in parallel with sensor control,….,depth data processing” indicating the digital camera includes RGB data and depth data being processed in parallel [together], therefore, the video streams as discussed comprises RGB stream, indicating a target image); and sending the target image to a playback device for playback (FIG. 4 illustrates, in step 490, a repeating of a procedure for both left and right displays, indicating a playback on a display device).
Regarding claim 18, Rowell in view of Gaddy and Wu, wherein Rowel discloses the electronic device of claim 12, wherein determining, according to the perspective information, the target image (Column 10, lines 10-26, discloses “example sensors that may be incorporated in the digital camera device include….visible light sensors (e.g., RGB cameras);” moreover, Column 42, lines 42-53, discloses “the auto re-calibration subsystem can work on any digital camera device….to ensure digital re-calibration process can run in parallel with sensor control,….,depth data processing” indicating the digital camera includes RGB data and depth data being processed in parallel [together], therefore, the video streams as discussed comprises RGB stream, indicating a target image), comprises: configuring, according to the perspective information, a virtual camera (FIG. 4 illustrates, at step 490, a procedure on a VR headset [virtual camera], as the result of the previous steps 410-480 including the determination as discussed); and determining an image photographed by the virtual camera as a target image (Col. 10, lines 1-10, discloses “having multiple cameras….used to capture the visual aspects of a scene” indicating perspectives [visual aspects] of at least two cameras of a same scene, the camera of a VR headset would be a virtual camera and the image captured by the system would be a target image).
Claims 2-3 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Adam Rowell et. al. (“US 10,602,126 B2” hereinafter as “Rowell”) in view of William L. Gaddy et. al. (“US 8,988,502” hereinafter as “Gaddy”) further in view of Chen Wu et. al. (“US 2014/0118494 A1” hereinafter as “Wu”) and Carlo Dal Mutto et. al. (“US 2025/0047983 A1” hereinafter as “Mutto”).
Regarding claim 2, Rowell in view of Gaddy and Wu, wherein Rowel discloses the method of claim 1, wherein the at least two cameras comprise a master camera and a plurality of slave cameras (Col. 10, lines 1-10, discloses “having multiple cameras….used to capture the visual aspects of a scene” indicating perspectives [visual aspects] of at least two cameras of a same scene, any of which is analogous to the recited master camera and the others to be slave cameras); to align pose of point cloud streams from the perspectives of the plurality of slave cameras with pose of point cloud streams from a perspective of the master camera (Col. 29, lines 44-52, discloses “align an image captured by one camera module to another image captured by another camera module;” furthermore, Col. 40, lines 53-67, discloses “over time, the position of one or more camera modules may shift…calibration process rectification by offsetting the alignment of images and videos generated by left and right camera modules” indicating a registering between video streams to align the views of the cameras [perspectives of the cameras] according to offsetting [preset registration information] according to the poise of the cameras, Col. 3, lines 41-60).
However, Rowell in view of Gaddy and Wu does not explicitly disclose the preset registration information is a plurality of pose transformation matrices between the plurality of slave cameras and the master camera, and registering, according to the preset registration information, the depth video streams from the perspectives of the at least two cameras, comprises: extracting point cloud streams from the perspectives of the at least two cameras corresponding to the depth video streams from the perspectives of the at least two cameras in a one-to-one correspondence; and performing, according to the plurality of pose transformation matrices, pose transformation on point cloud streams from perspectives of the plurality of slave cameras, to align pose of transformed point cloud streams from the perspectives of the plurality of slave cameras with pose of point cloud streams from a perspective of the master camera.
In the same field of calibration of cameras (Abstract, Mutto), Mutto discloses the preset registration information is a plurality of pose transformation matrices between the plurality of slave cameras and the master camera (Par. [0069] discloses “camera calibration information can provide information to rectify input images….in the master and in the slave image” which are obtained from, Par. [0065], “master camera and one or more slave cameras;” wherein, Par. [0003], “camera calibration…is the process of estimating the parameters…the parameters include: intrinsic parameters…and extrinsic parameters which denote coordinate system transformations between 3D world coordinates and 3D camera coordinates;” importantly, Par. [0091] discloses “relative pose may be defined as 3D rigid transformation that would map the location and orientation…..relative pose may include two transformations” which indicates a transformation here is in matrix/vector form [including two or more values as in a term]; these paragraphs, together, teach that a camera calibration process includes aligning camera poses by using transformation matrices of a master camera and a plurality of slave cameras), and registering, according to the preset registration information, the depth video streams from the perspectives of the at least two cameras (Par. [0056] discloses “the depth cameras….include at least two standard 2D cameras that have overlapping fields of view….the 2D cameras may be substantially parallel such that the two cameras image substantially the same scene, albeit from slightly different perspectives;” furthermore, Par. [0118] discloses “accurate perspective with respect to the shown coordinate axes….calibrate the camera groups…by placing a calibration target in a location that is simultaneously within the fields of view of at least one camera….each camera group may be calibrated with respect to a reference global coordinate system” indicating that the camera calibration process is based on fields of view of the cameras [perspectives] to register the cameras capturing the same scene and aligned images), comprises: extracting point cloud streams from the perspectives of the at least two cameras corresponding to the depth video streams (Par. [0117] discloses “calibrating the cameras…determine an offset or transformation between chunks of data…of the camera groups….chunks that are captured by different camera groups…(e.g., if the chunks corresponded to point clouds representing different parts of the target object)” indicating extracting point cloud streams [chunks] from the perspectives of the cameras since the chunks corresponds to different portions [different perspectives] of the target object captured by different fields of view of the cameras) from the perspectives of the at least two cameras in a one-to-one correspondence (Par. [0118] discloses “camera groups…positioned at different portions of a manufacturing line to image objects….calibrate the camera groups 130ABC and 130DEF with respect to one another by calibration” according to FIG. 7, therefore, indicating that the different perspective of the cameras are in one-to-one correspondence [calibrate the camera groups with respect to one another]); and performing, according to the plurality of pose transformation matrices, pose transformation on point cloud streams from perspectives of the plurality of slave cameras (Par. [0117] discloses “determine an offset or transformation between chunks of data captured” indicating an alignment process using the offset based on the transformation, being a matrix as discussed previously, between chunks [point cloud streams as discussed] which are obtained from different points of view/perspectives of the cameras; wherein Par. [0095] discloses “third relative pose may be the composition of the transformation from the pose of the third camera to the pose of the second camera” indicating the pose transformation), to align pose of transformed point cloud streams from the perspectives of the plurality of slave cameras with pose of point cloud streams from a perspective of the master camera (as discussed previously, the calibration to align perspectives of the different cameras using an offset/transformation, therefore, the aligning is to align pose the transformed point cloud streams accordingly).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell in view of Gaddy and Wu’s method, wherein Rowell’s registering, according to a preset registration information, depth video streams from perspectives of at least two cameras can be modified to comprise extracting point cloud streams from the perspectives of the cameras corresponding to the depth video streams from the perspectives of the at least two cameras in a one-to-one correspondence and performing, according to a plurality of pose transformation matrices, pose transformation on point cloud streams from perspectives of a plurality of slave cameras, to align pose of transformed point cloud streams from the perspectives of the slave cameras with pose of point cloud streams from perspectives of a master camera as taught by Mutto as discussed in the mapping above.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to use such method to calibrate camera group more accurately (Par. [0119], Mutto).
Regarding claim 3, Rowell in view of Gaddy further in view of Wu and Mutto discloses the method of claim 2.
However, Rowell in view of Gaddy further in view of Wu does not explicitly disclose wherein acquiring the plurality of pose transformation matrices between the plurality of slave cameras and the master camera comprises: controlling the plurality of slave cameras and the master camera to photograph a calibration object to acquire a plurality of pictures containing the calibration object; performing feature detection on the plurality of pictures containing the calibration object to acquire pose information of the calibration object in each picture of the plurality of pictures containing the calibration object; and determining, according to the pose information of the calibration object in the each picture, a plurality of pose transformation matrices between the plurality of slave cameras and the master camera where a pose transformation matrix of the plurality of pose transformation matrices exists between a respective slave camera of the plurality of slave cameras and the master camera; or acquiring the plurality of pose transformation matrices between the plurality of slave cameras and the master camera comprises: acquiring, by adopting a set algorithm, the plurality of pose transformation matrices between the plurality of slave cameras and the master camera.
In the same field of calibration of cameras (Abstract, Mutto), Mutto discloses wherein acquiring the plurality of pose transformation matrices between the plurality of slave cameras and the master camera comprises:
controlling the plurality of slave cameras and the master camera to photograph a calibration object to acquire a plurality of pictures containing the calibration object; performing feature detection on the plurality of pictures containing the calibration object to acquire pose information of the calibration object in each picture of the plurality of pictures containing the calibration object; and determining, according to the pose information of the calibration object in the each picture, a plurality of pose transformation matrices between the plurality of slave cameras and the master camera where a pose transformation matrix of the plurality of pose transformation matrices exists between a respective slave camera of the plurality of slave cameras and the master camera;
or (“or” indicates a selection, only one of the options is the instant scope of the claim; therefore, the examiner selects the following limitation of “acquiring the plurality of pose transformation matrices….comprises acquiring, adopting a set algorithm….master camera” for mapping),
acquiring the plurality of pose transformation matrices between the plurality of slave cameras and the master camera comprises (Par. [0069] discloses “camera calibration information can provide information to rectify input images….in the master and in the slave image” which are obtained from, Par. [0065], “master camera and one or more slave cameras;” wherein, Par. [0003], “camera calibration…is the process of estimating the parameters…the parameters include: intrinsic parameters…and extrinsic parameters which denote coordinate system transformations between 3D world coordinates and 3D camera coordinates;” importantly, Par. [0091] discloses “relative pose may be defined as 3D rigid transformation that would map the location and orientation…..relative pose may include two transformations” which indicates a transformation here is in matrix/vector form [including two or more values as in a term]; these paragraphs, together, teach that a camera calibration process includes aligning camera poses by using transformation matrices of a master camera and a plurality of slave cameras):
acquiring, by adopting a set algorithm (Par. [0005] discloses “algorithms that involves the use of the two or more images, such as image stitching and stereo imaging, may require….alignment process may involve applying transformations to the images based on the known camera poses” indicating a set algorithm), the plurality of pose transformation matrices between the plurality of slave cameras and the master camera (Par. [0117] discloses “determine an offset or transformation between chunks of data captured” indicating an alignment process using the offset based on the transformation, being a matrix as discussed previously, between chunks [point cloud streams as discussed] which are obtained from different points of view/perspectives of the cameras; wherein Par. [0095] discloses “third relative pose may be the composition of the transformation from the pose of the third camera to the pose of the second camera” indicating the pose transformation).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell in view of Gaddy and Wu’s method, wherein Rowell’s registering, according to a preset registration information, depth video streams from perspectives of at least two cameras can be modified to be based on a process of acquiring the plurality of pose transformation matrices between the plurality of slave cameras and the master camera which comprises acquiring, by adopting a set algorithm, the plurality of pose transformation matrices between the plurality of slave cameras and the master camera as taught by Mutto as discussed in the mapping above.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to use such method to calibrate camera group more accurately (Par. [0119], Mutto).
Regarding claim 14, Rowell in view of Gaddy and Wu, wherein Rowel discloses the electronic device of claim 12, wherein the at least two cameras comprise a master camera and a plurality of slave cameras (Col. 10, lines 1-10, discloses “having multiple cameras….used to capture the visual aspects of a scene” indicating perspectives [visual aspects] of at least two cameras of a same scene, any of which is analogous to the recited master camera and the others to be slave cameras); to align pose of point cloud streams from the perspectives of the plurality of slave cameras with pose of point cloud streams from a perspective of the master camera (Col. 29, lines 44-52, discloses “align an image captured by one camera module to another image captured by another camera module;” furthermore, Col. 40, lines 53-67, discloses “over time, the position of one or more camera modules may shift…calibration process rectification by offsetting the alignment of images and videos generated by left and right camera modules” indicating a registering between video streams to align the views of the cameras [perspectives of the cameras] according to offsetting [preset registration information] according to the poise of the cameras, Col. 3, lines 41-60).
However, Rowell in view of Gaddy and Wu does not explicitly disclose the preset registration information is a plurality of pose transformation matrices between the plurality of slave cameras and the master camera, and registering, according to the preset registration information, the depth video streams from the perspectives of the at least two cameras, comprises: extracting point cloud streams from the perspectives of the at least two cameras corresponding to the depth video streams from the perspectives of the at least two cameras in a one-to-one correspondence; and performing, according to the plurality of pose transformation matrices, pose transformation on point cloud streams from perspectives of the plurality of slave cameras, to align pose of transformed point cloud streams from the perspectives of the plurality of slave cameras with pose of point cloud streams from a perspective of the master camera.
In the same field of calibration of cameras (Abstract, Mutto), Mutto discloses the preset registration information is a plurality of pose transformation matrices between the plurality of slave cameras and the master camera (Par. [0069] discloses “camera calibration information can provide information to rectify input images….in the master and in the slave image” which are obtained from, Par. [0065], “master camera and one or more slave cameras;” wherein, Par. [0003], “camera calibration…is the process of estimating the parameters…the parameters include: intrinsic parameters…and extrinsic parameters which denote coordinate system transformations between 3D world coordinates and 3D camera coordinates;” importantly, Par. [0091] discloses “relative pose may be defined as 3D rigid transformation that would map the location and orientation…..relative pose may include two transformations” which indicates a transformation here is in matrix/vector form [including two or more values as in a term]; these paragraphs, together, teach that a camera calibration process includes aligning camera poses by using transformation matrices of a master camera and a plurality of slave cameras), and registering, according to the preset registration information, the depth video streams from the perspectives of the at least two cameras (Par. [0056] discloses “the depth cameras….include at least two standard 2D cameras that have overlapping fields of view….the 2D cameras may be substantially parallel such that the two cameras image substantially the same scene, albeit from slightly different perspectives;” furthermore, Par. [0118] discloses “accurate perspective with respect to the shown coordinate axes….calibrate the camera groups…by placing a calibration target in a location that is simultaneously within the fields of view of at least one camera….each camera group may be calibrated with respect to a reference global coordinate system” indicating that the camera calibration process is based on fields of view of the cameras [perspectives] to register the cameras capturing the same scene and aligned images), comprises: extracting point cloud streams from the perspectives of the at least two cameras corresponding to the depth video streams (Par. [0117] discloses “calibrating the cameras…determine an offset or transformation between chunks of data…of the camera groups….chunks that are captured by different camera groups…(e.g., if the chunks corresponded to point clouds representing different parts of the target object)” indicating extracting point cloud streams [chunks] from the perspectives of the cameras since the chunks corresponds to different portions [different perspectives] of the target object captured by different fields of view of the cameras) from the perspectives of the at least two cameras in a one-to-one correspondence (Par. [0118] discloses “camera groups…positioned at different portions of a manufacturing line to image objects….calibrate the camera groups 130ABC and 130DEF with respect to one another by calibration” according to FIG. 7, therefore, indicating that the different perspective of the cameras are in one-to-one correspondence [calibrate the camera groups with respect to one another]); and performing, according to the plurality of pose transformation matrices, pose transformation on point cloud streams from perspectives of the plurality of slave cameras (Par. [0117] discloses “determine an offset or transformation between chunks of data captured” indicating an alignment process using the offset based on the transformation, being a matrix as discussed previously, between chunks [point cloud streams as discussed] which are obtained from different points of view/perspectives of the cameras; wherein Par. [0095] discloses “third relative pose may be the composition of the transformation from the pose of the third camera to the pose of the second camera” indicating the pose transformation), to align pose of transformed point cloud streams from the perspectives of the plurality of slave cameras with pose of point cloud streams from a perspective of the master camera (as discussed previously, the calibration to align perspectives of the different cameras using an offset/transformation, therefore, the aligning is to align pose the transformed point cloud streams accordingly).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell in view of Gaddy and Wu’s method, wherein Rowell’s registering, according to a preset registration information, depth video streams from perspectives of at least two cameras can be modified to comprise extracting point cloud streams from the perspectives of the cameras corresponding to the depth video streams from the perspectives of the at least two cameras in a one-to-one correspondence and performing, according to a plurality of pose transformation matrices, pose transformation on point cloud streams from perspectives of a plurality of slave cameras, to align pose of transformed point cloud streams from the perspectives of the slave cameras with pose of point cloud streams from perspectives of a master camera as taught by Mutto as discussed in the mapping above.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to use such method to calibrate camera group more accurately (Par. [0119], Mutto).
Regarding claim 15, Rowell in view of Gaddy further in view of Wu and Mutto discloses the electronic device of claim 14.
However, Rowell in view of Gaddy further in view of Wu does not explicitly disclose wherein acquiring the plurality of pose transformation matrices between the plurality of slave cameras and the master camera comprises: controlling the plurality of slave cameras and the master camera to photograph a calibration object to acquire a plurality of pictures containing the calibration object; performing feature detection on the plurality of pictures containing the calibration object to acquire pose information of the calibration object in each picture of the plurality of pictures containing the calibration object; and determining, according to the pose information of the calibration object in the each picture, a plurality of pose transformation matrices between the plurality of slave cameras and the master camera where a pose transformation matrix of the plurality of pose transformation matrices exists between a respective slave camera of the plurality of slave cameras and the master camera; or acquiring the plurality of pose transformation matrices between the plurality of slave cameras and the master camera comprises: acquiring, by adopting a set algorithm, the plurality of pose transformation matrices between the plurality of slave cameras and the master camera.
In the same field of calibration of cameras (Abstract, Mutto), Mutto discloses wherein acquiring the plurality of pose transformation matrices between the plurality of slave cameras and the master camera comprises:
controlling the plurality of slave cameras and the master camera to photograph a calibration object to acquire a plurality of pictures containing the calibration object; performing feature detection on the plurality of pictures containing the calibration object to acquire pose information of the calibration object in each picture of the plurality of pictures containing the calibration object; and determining, according to the pose information of the calibration object in the each picture, a plurality of pose transformation matrices between the plurality of slave cameras and the master camera where a pose transformation matrix of the plurality of pose transformation matrices exists between a respective slave camera of the plurality of slave cameras and the master camera;
or (“or” indicates a selection, only one of the options is the instant scope of the claim; therefore, the examiner selects the following limitation of “acquiring the plurality of pose transformation matrices….comprises acquiring, adopting a set algorithm….master camera” for mapping),
acquiring the plurality of pose transformation matrices between the plurality of slave cameras and the master camera comprises (Par. [0069] discloses “camera calibration information can provide information to rectify input images….in the master and in the slave image” which are obtained from, Par. [0065], “master camera and one or more slave cameras;” wherein, Par. [0003], “camera calibration…is the process of estimating the parameters…the parameters include: intrinsic parameters…and extrinsic parameters which denote coordinate system transformations between 3D world coordinates and 3D camera coordinates;” importantly, Par. [0091] discloses “relative pose may be defined as 3D rigid transformation that would map the location and orientation…..relative pose may include two transformations” which indicates a transformation here is in matrix/vector form [including two or more values as in a term]; these paragraphs, together, teach that a camera calibration process includes aligning camera poses by using transformation matrices of a master camera and a plurality of slave cameras):
acquiring, by adopting a set algorithm (Par. [0005] discloses “algorithms that involves the use of the two or more images, such as image stitching and stereo imaging, may require….alignment process may involve applying transformations to the images based on the known camera poses” indicating a set algorithm), the plurality of pose transformation matrices between the plurality of slave cameras and the master camera (Par. [0117] discloses “determine an offset or transformation between chunks of data captured” indicating an alignment process using the offset based on the transformation, being a matrix as discussed previously, between chunks [point cloud streams as discussed] which are obtained from different points of view/perspectives of the cameras; wherein Par. [0095] discloses “third relative pose may be the composition of the transformation from the pose of the third camera to the pose of the second camera” indicating the pose transformation).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell in view of Gaddy and Wu’s method, wherein Rowell’s registering, according to a preset registration information, depth video streams from perspectives of at least two cameras can be modified to be based on a process of acquiring the plurality of pose transformation matrices between the plurality of slave cameras and the master camera which comprises acquiring, by adopting a set algorithm, the plurality of pose transformation matrices between the plurality of slave cameras and the master camera as taught by Mutto as discussed in the mapping above.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to use such method to calibrate camera group more accurately (Par. [0119], Mutto).
Claims 4 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Adam Rowell et. al. (“US 10,602,126 B2” hereinafter as “Rowell”) in view of William L. Gaddy et. al. (“US 8,988,502” hereinafter as “Gaddy”) further in view of Chen Wu et. al. (“US 2014/0118494 A1” hereinafter as “Wu”) and Carlo Dal Mutto et. al. (“US 2025/0047983 A1” hereinafter as “Mutto”) and Mourad Boufarguine (“US 2016/0189358 A1” hereinafter as “Boufarguine”).
Regarding claim 4, Rowell in view of Gaddy further in view of Wu and Mutto, wherein Rowell discloses the method of claim 2, wherein performing, according to the registered depth video streams from the perspectives of the at least two cameras, the three-dimensional reconstruction to obtain the 3D video (Col. 3, lines 41-60, discloses “the digital camera device cannot function properly without effective calibration and rectification techniques….delivering output of every digital camera device….including 3D content generation, scene reconstruction” indicating using of the result of the camera calibration [registered depth video streams as discussed previously] to perform scene reconstruction [3D reconstruction of the video data]); adopting a set three-dimensional reconstruction algorithm to perform (the 3D reconstruction process as discussed is analogous to the 3D reconstruction algorithm as claimed) the reconstruction from the perspectives of the plurality of slave cameras and the point cloud streams (Col. 10, lines 1-10, discloses “having multiple cameras….used to capture the visual aspects of a scene” indicating perspectives [visual aspects] of at least two cameras of a same scene, any of which is analogous to the recited master camera and the others to be slave cameras from the perspective of the master camera (Col. 29, lines 44-52, discloses “align an image captured by one camera module to another image captured by another camera module;” furthermore, Col. 40, lines 53-67, discloses “over time, the position of one or more camera modules may shift…calibration process rectification by offsetting the alignment of images and videos generated by left and right camera modules” indicating a registering between video streams to align the views of the cameras [perspectives of the cameras] according to offsetting [preset registration information] according to the poise of the cameras, Col. 3, lines 41-60) to obtain the 3D video (Col. 3, lines 41-60, discloses “the digital camera device cannot function properly without effective calibration and rectification techniques….delivering output of every digital camera device….including 3D content generation, scene reconstruction” indicating using of the result of the camera calibration [registered depth video streams as discussed previously] to perform scene reconstruction [3D reconstruction of the video data]).
However, Rowell in view of Gaddy further in view of Wu and Mutto does not explicitly disclose adopting a set three-dimensional reconstruction algorithm to perform fusion and surface estimation on the transformed point cloud streams.
In the same field of depth camera calibration (Abstract, Boufarguine), Boufarguine discloses adopting a set three-dimensional reconstruction algorithm to perform fusion and surface estimation (Par. [0005] discloses “a three-dimensional reconstruction 3DR for physical object taking the depth maps as inputs. Several suitable reconstruction algorithms are known in the art, e.g. KinectFusion: Real-Time Dense Surface Mapping and Tracking” indicating a 3D reconstruction performing fusion and surface estimation) on the transformed point cloud streams (Par. [0005] discloses “a depth camera DC can be used to acquire multiple depth maps….reconstruction…taking depth maps as inputs” indicating the reconstruction is on the point cloud data; moreover, Par. [0072] discloses “final rigid transformation between the point clouds” indicating the point cloud data being used is transformed point cloud data which is used for the subsequential reconstruction disclosed in Par. [0074]).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell in view of Gaddy further in view of Wu and Mutto’s method, wherein Rowell’s performing, according to the registered depth video streams from the perspectives of the at least two cameras, the three-dimensional reconstruction to obtain the 3D video can be modified to be based on performing fusion and surface estimation on the transformed point cloud streams as taught by Boufarguine as discussed in the mapping above.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to use such method to calibrate camera in a better approach suited for 3D reconstruction basing on fusion and surface tracking for the 3D reconstruction such as have been discussed (Par. [0015], Boufarguine).
Regarding claim 16, Rowell in view of Gaddy further in view of Wu and Mutto, wherein Rowell discloses the electronic device of claim 14, wherein performing, according to the registered depth video streams from the perspectives of the at least two cameras, the three-dimensional reconstruction to obtain the 3D video (Col. 3, lines 41-60, discloses “the digital camera device cannot function properly without effective calibration and rectification techniques….delivering output of every digital camera device….including 3D content generation, scene reconstruction” indicating using of the result of the camera calibration [registered depth video streams as discussed previously] to perform scene reconstruction [3D reconstruction of the video data]); adopting a set three-dimensional reconstruction algorithm to perform (the 3D reconstruction process as discussed is analogous to the 3D reconstruction algorithm as claimed) the reconstruction from the perspectives of the plurality of slave cameras and the point cloud streams (Col. 10, lines 1-10, discloses “having multiple cameras….used to capture the visual aspects of a scene” indicating perspectives [visual aspects] of at least two cameras of a same scene, any of which is analogous to the recited master camera and the others to be slave cameras from the perspective of the master camera (Col. 29, lines 44-52, discloses “align an image captured by one camera module to another image captured by another camera module;” furthermore, Col. 40, lines 53-67, discloses “over time, the position of one or more camera modules may shift…calibration process rectification by offsetting the alignment of images and videos generated by left and right camera modules” indicating a registering between video streams to align the views of the cameras [perspectives of the cameras] according to offsetting [preset registration information] according to the poise of the cameras, Col. 3, lines 41-60) to obtain the 3D video (Col. 3, lines 41-60, discloses “the digital camera device cannot function properly without effective calibration and rectification techniques….delivering output of every digital camera device….including 3D content generation, scene reconstruction” indicating using of the result of the camera calibration [registered depth video streams as discussed previously] to perform scene reconstruction [3D reconstruction of the video data]).
However, Rowell in view of Gaddy further in view of Wu and Mutto does not explicitly disclose adopting a set three-dimensional reconstruction algorithm to perform fusion and surface estimation on the transformed point cloud streams.
In the same field of depth camera calibration (Abstract, Boufarguine), Boufarguine discloses adopting a set three-dimensional reconstruction algorithm to perform fusion and surface estimation (Par. [0005] discloses “a three-dimensional reconstruction 3DR for physical object taking the depth maps as inputs. Several suitable reconstruction algorithms are known in the art, e.g. KinectFusion: Real-Time Dense Surface Mapping and Tracking” indicating a 3D reconstruction performing fusion and surface estimation) on the transformed point cloud streams (Par. [0005] discloses “a depth camera DC can be used to acquire multiple depth maps….reconstruction…taking depth maps as inputs” indicating the reconstruction is on the point cloud data; moreover, Par. [0072] discloses “final rigid transformation between the point clouds” indicating the point cloud data being used is transformed point cloud data which is used for the subsequential reconstruction disclosed in Par. [0074]).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell in view of Gaddy further in view of Wu and Mutto’s method, wherein Rowell’s performing, according to the registered depth video streams from the perspectives of the at least two cameras, the three-dimensional reconstruction to obtain the 3D video can be modified to be based on performing fusion and surface estimation on the transformed point cloud streams as taught by Boufarguine as discussed in the mapping above.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to use such method to calibrate camera in a better approach suited for 3D reconstruction basing on fusion and surface tracking for the 3D reconstruction such as have been discussed (Par. [0015], Boufarguine).
Claims 7 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Adam Rowell et. al. (“US 10,602,126 B2” hereinafter as “Rowell”) in view of William L. Gaddy et. al. (“US 8,988,502” hereinafter as “Gaddy”) further in view of Chen Wu et. al. (“US 2014/0118494 A1” hereinafter as “Wu”) and Andraes Blassnig et. al. (“US 9,544,577 B2” hereinafter as “Blassnig”).
Regarding claim 7, Rowell in view of Gaddy and Wu, wherein Rowel discloses the method of claim 6, wherein determining the image photographed by the virtual camera as the target image (Col. 10, lines 1-10, discloses “having multiple cameras….used to capture the visual aspects of a scene” indicating perspectives [visual aspects] of at least two cameras of a same scene, the camera of a VR headset would be a virtual camera and the image captured by the system would be a target image).
However, Rowell in view of Gaddy and Wu does not explicitly disclose determining an intersection point of light emitted by the virtual camera and a nearest object as a pixel point in an image photographed by the virtual camera; determining two-dimensional coordinates of the intersection point in a map formed by a surface of the nearest object; and determining, according to the two-dimensional coordinates, a pixel value of the intersection point by adopting a set interpolation method.
In the same field of 3D image processing for virtual camera (Abstract, Blassnig), Blassnig discloses determining an intersection point of light emitted by the virtual camera (Column 3, lines 54-67 to Column 4, lines 1-3, disclose “fit this image into the predefined ray casting position…using ICP iterative closest point algorithm; moreover, the ray-casting is based on intersection determination according to Fig. 9, furthermore, Column 16, lines 8-10, discloses “calculation of the intersection by means of interpolation….between two successive sampling points along the line of sight” indicating a determination of intersection point of light emitted [line of sight]) and a nearest object as a pixel point in an image photographed by the virtual camera (the ICP process, as discussed in Column 4, lines 1-3, indicating a nearest point algorithm ICP; wherein the ICP process is laid out in Column 4, lines 5-67, wherein it’s based on the closest point of camera which can be understood to be the nearest object as pixel point in the image photographed by the camera such as disclosed in Column 3, lines 60-67, “scanning process and the parts of the object outside this model structure” indicating the object information being used for the ICP process); determining two-dimensional coordinates of the intersection point in a map (Column 12, lines 10-16, discloses the coordinate of the intersection in 2D being determined “z=f(x,y)” wherein, it’s process in the mapping of the ICP registration process according to Column 23, lines 1-5, “vertex map of the current depth image prior to using it for an ICP registration attempt” indicating the map formed by the intersection point of the surface of the nearest object in the ICP processing) formed by a surface of the nearest object (Column 23, lines 1-5, “vertex map of the current depth image prior to using it for an ICP registration attempt…to the stored model surface” indicating the map formed by the intersection point of the surface of the nearest object in the ICP processing); and determining, according to the two-dimensional coordinates, a pixel value of the intersection point by adopting a set interpolation method (Column 16, lines 8-10, discloses “calculation of the intersection by means of interpolation….between two successive sampling points along the line of sight” indicating the determining of the 2D coordinates of the pixel value is based on interpolation method such as shown in equation 12-16).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell in view of Gaddy and Wu’s method, wherein Rowell’s performing determining the image photographed by the virtual camera as the target image can be modified to be based on determining an intersection point of light emitted by the virtual camera and a nearest object as a pixel point in an image photographed by the virtual camera; determining two-dimensional coordinates of the intersection point in a map formed by a surface of the nearest object; and determining, according to the two-dimensional coordinates, a pixel value of the intersection point by adopting a set interpolation method as taught by Blassnig as discussed in the mapping above.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to interpolation of points determined based on intersection and nearest object in 3D information processing according to 2D coordinates to perform the processing in an improved approach (Column 14, lines 1-14, Blassnig).
Regarding claim 19, Rowell in view of Gaddy and Wu, wherein Rowel discloses the electronic device of claim 18, wherein determining the image photographed by the virtual camera as the target image (Col. 10, lines 1-10, discloses “having multiple cameras….used to capture the visual aspects of a scene” indicating perspectives [visual aspects] of at least two cameras of a same scene, the camera of a VR headset would be a virtual camera and the image captured by the system would be a target image).
However, Rowell in view of Gaddy and Wu does not explicitly disclose determining an intersection point of light emitted by the virtual camera and a nearest object as a pixel point in an image photographed by the virtual camera; determining two-dimensional coordinates of the intersection point in a map formed by a surface of the nearest object; and determining, according to the two-dimensional coordinates, a pixel value of the intersection point by adopting a set interpolation method.
In the same field of 3D image processing for virtual camera (Abstract, Blassnig), Blassnig discloses determining an intersection point of light emitted by the virtual camera (Column 3, lines 54-67 to Column 4, lines 1-3, disclose “fit this image into the predefined ray casting position…using ICP iterative closest point algorithm; moreover, the ray-casting is based on intersection determination according to Fig. 9, furthermore, Column 16, lines 8-10, discloses “calculation of the intersection by means of interpolation….between two successive sampling points along the line of sight” indicating a determination of intersection point of light emitted [line of sight]) and a nearest object as a pixel point in an image photographed by the virtual camera (the ICP process, as discussed in Column 4, lines 1-3, indicating a nearest point algorithm ICP; wherein the ICP process is laid out in Column 4, lines 5-67, wherein it’s based on the closest point of camera which can be understood to be the nearest object as pixel point in the image photographed by the camera such as disclosed in Column 3, lines 60-67, “scanning process and the parts of the object outside this model structure” indicating the object information being used for the ICP process); determining two-dimensional coordinates of the intersection point in a map (Column 12, lines 10-16, discloses the coordinate of the intersection in 2D being determined “z=f(x,y)” wherein, it’s process in the mapping of the ICP registration process according to Column 23, lines 1-5, “vertex map of the current depth image prior to using it for an ICP registration attempt” indicating the map formed by the intersection point of the surface of the nearest object in the ICP processing) formed by a surface of the nearest object (Column 23, lines 1-5, “vertex map of the current depth image prior to using it for an ICP registration attempt…to the stored model surface” indicating the map formed by the intersection point of the surface of the nearest object in the ICP processing); and determining, according to the two-dimensional coordinates, a pixel value of the intersection point by adopting a set interpolation method (Column 16, lines 8-10, discloses “calculation of the intersection by means of interpolation….between two successive sampling points along the line of sight” indicating the determining of the 2D coordinates of the pixel value is based on interpolation method such as shown in equation 12-16).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell in view of Gaddy and Wu’s method, wherein Rowell’s performing determining the image photographed by the virtual camera as the target image can be modified to be based on determining an intersection point of light emitted by the virtual camera and a nearest object as a pixel point in an image photographed by the virtual camera; determining two-dimensional coordinates of the intersection point in a map formed by a surface of the nearest object; and determining, according to the two-dimensional coordinates, a pixel value of the intersection point by adopting a set interpolation method as taught by Blassnig as discussed in the mapping above.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to interpolation of points determined based on intersection and nearest object in 3D information processing according to 2D coordinates to perform the processing in an improved approach (Column 14, lines 1-14, Blassnig).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Adam Rowell et. al. (“US 10,602,126 B2” hereinafter as “Rowell”) in view of William L. Gaddy et. al. (“US 8,988,502” hereinafter as “Gaddy”) further in view of Chen Wu et. al. (“US 2014/0118494 A1” hereinafter as “Wu”) and William E. Gardner (“US 2007/0076265 A1” hereinafter as “Gardner”).
Regarding claim 20, Rowell in view of Gaddy and Wu discloses the electronic device of claim 12.
However, Rowell in view of Gaddy and Wu does not explicitly disclose wherein evenly distributing the depth information stream to the RGB channel, comprises: evenly distributing bit data corresponding to the depth information stream to high bits of the RGB channels.
In the same field of depth data processing (Title and abstract, Gardner), Gardner discloses wherein evenly distributing the depth information stream to the RGB channel (Par. [0036] discloses “the N-bit per channel data is quantized to M-bit per channel data….uniform intervals” which indicates a quantizing [distributing] using uniform intervals [evenly distributing or uniform interval method of distribution]; moreover, Par. [0031] discloses “each channel…perform bit depth reduction” indicating the quantizing is perform on depth information of each RGB channel), comprises: evenly distributing bit data corresponding to the depth information stream to high bits of the RGB channels (Par. [0036] discloses “the N-bit per channel data is quantized to M-bit per channel data….uniform intervals” which indicates the uniform interval method of quantizing which is analogous to evenly distributing bit data corresponding to the depth information, Par. [0031] discloses “each channel…perform bit depth reduction” indicating the quantizing is perform on depth information of each RGB channel; furthermore, going from N-bit to M-bit indicating going to high bits of the RGB channels accordingly).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Rowell’s method of acquiring depth video streams from perspective of at least two cameras of a same scene, wherein each of the depth video streams comprises a Red-Green-Blue stream and depth information stream; wherein Rowell’s acquiring of the RGB stream which is to a cloud server through a respective one of RGB channels can be modified to have the depth information stream is distributed to the RGB channels and sent to the cloud server through the RGB channels as taught by Gaddy and the depth information stream is evenly distributed to the RGB channels, wherein evenly distributing the depth information stream to the RGB channel, comprises: evenly distributing bit data corresponding to the depth information stream to high bits of the RGB channels as taught by Gardner.
Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to process depth data using such quantizing method as discussed to reduce data for lesser load for processing efficiently (Abstract, Gardner).
Pertinent Prior Art(s)
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Batia Mach Shepherd et. al., “US 2017/0249783 A1” discloses a 3D image processing for display (Abstract) based on point cloud transformation processing according to extreme point of object according to Par. [0016] for 3D reconstruction of image using a method of camera calibration according to Par. [0022].
Anthony Tran et. al., “US 9,872,010 B2” discloses a 3D image data processing (Abstract) for camera calibration using interocular distance between cameras according to Column 7, lines 1-17, which is for the process of 3D reconstruction according to Column 15, lines 63-67.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHUONG HAU CAI whose telephone number is (571)272-9424. The examiner can normally be reached M-F 8:30 am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHUONG HAU CAI/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights