DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2, 4-12, 15-16, 18, 20-23 are rejected under 35 U.S.C. 103 as being unpatentable over Lewis et al. (US 9294757 B1) in view of Ustarroz-Calonge et al. (US 20220014792 A1).
Regarding claim 16. Lewis discloses A device, comprising at least one memory, at least one processor and a computer program that is stored in the at least one memory and executable on the at least one processor (column 18 lines 16-22, The example computing device 900 includes a processing device (e.g., a processor) 902, and a data storage device 918; column 18 lines 49-52, The data storage device 918 may include a computer-readable storage medium 928 on which is stored one or more sets of instructions (e.g., image module 926) embodying any one or more of the methodologies or functions described herein), wherein when the at least one processor executes the computer program, the at least one processor implements an image display method, wherein the image display method is applied to a client (abstract, The 3D video is provided to a user for viewing the object), and comprises:
receiving a two-dimensional image set sent by a server, wherein the two-dimensional image set is used for recording two-dimensional images of a three-dimensional object model at a plurality of different viewpoints (column 3 lines 6-9, A server may process the arrays of images and generate a 3D video of the object. The 3D video may allow a user to view a video of the object from multiple viewpoints);
in response to a display instruction for the three-dimensional object model at a target viewpoint, receiving a two-dimensional target image corresponding to the target viewpoint from the two-dimensional image set (column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device)); and
displaying the two-dimensional target image (column 1 lines 46-53, The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device); column 5 lines 15-17, The 3D video may allow a user to view the object 115 from any of the viewpoints of the cameras 110A through 110P); wherein the image processing method is applied to the server (column 3 lines 6-9, A server may process the arrays of images and generate a 3D video of the object), and comprises:
acquiring the two-dimensional image set generated by the three-dimensional object model at the plurality of different viewpoints (column 3 lines 6-9, A server may process the arrays of images and generate a 3D video of the object. The 3D video may allow a user to view a video of the object from multiple viewpoints); and
sending the two-dimensional image set to the client, so that the client displays the two-dimensional target image corresponding to the target viewpoint (column 3 lines 6-9, A server may process the arrays of images and generate a 3D video of the object. The 3D video may allow a user to view a video of the object from multiple viewpoints; column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device)).
However, Lewis doesn’t explicitly disclose parsing a two-dimensional target image corresponding to the target viewpoint from the two-dimensional image set.
Ustarroz-Calonge discloses a computing device may present decoded video using a display device ([0027] the user interface 130 may be an audio-visual display device, and the computing device 100 may present audio, such as decoded audio, using the user interface 130 audio-visual display device, such as in conjunction with displaying video, such as decoded video), wherein the decoded video are generated by parsing a two-dimensional target image (abstract, decoding, from multiple color channels of a bitstream, color channel values for an encoded image, decoding, from an alpha channel of the bitstream, alpha channel values for the encoded image; [0099] the transmitting computing and communication device 100A can be implemented on a server, and the receiving computing and communication device 100B can be implemented on a device separate from the server, the transmitting computing and communication device 100A can encode content using an encoder 400 into an encoded video signal and transmit the encoded video signal to the communications device, the communications device can then decode the encoded video signal using a decoder 500).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the inventions of Lewis and Ustarroz-Calonge, to encode the two-dimensional image set and decode/parse a two-dimensional target image corresponding to the target viewpoint from the two-dimensional image set, in order to save storage and more efficiently transmit the two-dimensional image set (Ustarroz-Calonge [0001] an image can be rendered faster by reducing the total data size of the image using compression and decompression techniques).
Regarding claim 18. Lewis in view of Ustarroz-Calonge discloses The device according to claim 16, wherein the parsing a two-dimensional target image corresponding to the target viewpoint from the two-dimensional image set, comprises:
acquiring encoded data corresponding to the two-dimensional target image at the target viewpoint from the two-dimensional image set (Ustarroz-Calonge [0099] the transmitting computing and communication device 100A can be implemented on a server, and the receiving computing and communication device 100B can be implemented on a device separate from the server, the transmitting computing and communication device 100A can encode content using an encoder 400 into an encoded video signal and transmit the encoded video signal to the communications device; Lewis column 3 lines 6-9, A server may process the arrays of images and generate a 3D video of the object. The 3D video may allow a user to view a video of the object from multiple viewpoints; column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device)); and
parsing the encoded data to acquire the two-dimensional target image corresponding to the target viewpoint (Ustarroz-Calonge [0099] the communications device can then decode the encoded video signal using a decoder 500; Lewis column 3 lines 6-9, A server may process the arrays of images and generate a 3D video of the object. The 3D video may allow a user to view a video of the object from multiple viewpoints; column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device)).
The same motivation has been stated in claim 16.
Regarding claim 20. Lewis in view of Ustarroz-Calonge discloses The device according to claim 18, wherein the encoded data comprises second attribute information of the two-dimensional images, and the second attribute information is used to label position information of different channels of the two-dimensional images in the encoded data (Lewis column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device) (position information is comprised in the image arrays in order to identify the image arrays); Ustarroz-Calonge [0099] the transmitting computing and communication device 100A can encode content using an encoder 400 into an encoded video signal; Ustarroz-Calonge [0064] At least a portion of an image decoding may be determined or identified from an encoded bitstream at 602. The portion may include a current block or other image portion. The portion may correspond to a respective block within one or more color channels including respective color plane data, an alpha channel including alpha plane data, or both, of the encoded image); and
accordingly, the parsing the encoded data to acquire the two-dimensional target image corresponding to the target viewpoint, comprises:
acquiring position information of a color channel of the two-dimensional target image in the encoded data and position information of a transparency channel of the two-dimensional target image in the encoded data from the second attribute information (Lewis column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device) (position information is comprised in the image arrays in order to identify the image arrays); Ustarroz-Calonge [0064] At least a portion of an image decoding may be determined or identified from an encoded bitstream at 602. The portion may include a current block or other image portion. The portion may correspond to a respective block within one or more color channels including respective color plane data, an alpha channel including alpha plane data, or both, of the encoded image);
acquiring a color channel code corresponding to the target viewpoint from the encoded data according to the position information of the color channel of the two-dimensional target image in the encoded data (Lewis column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device) (position information is comprised in the image arrays in order to identify the image arrays); Ustarroz-Calonge [0064] At least a portion of an image decoding may be determined or identified from an encoded bitstream at 602. The portion may include a current block or other image portion. The portion may correspond to a respective block within one or more color channels including respective color plane data, an alpha channel including alpha plane data, or both, of the encoded image);
acquiring a transparency channel code corresponding to the target viewpoint from the encoded data according to the position information of the transparency channel of the two-dimensional target image in the encoded data (Lewis column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device) (position information is comprised in the image arrays in order to identify the image arrays); Ustarroz-Calonge [0064] At least a portion of an image decoding may be determined or identified from an encoded bitstream at 602. The portion may include a current block or other image portion. The portion may correspond to a respective block within one or more color channels including respective color plane data, an alpha channel including alpha plane data, or both, of the encoded image); and
fusing the color channel code and the transparency channel code to generate the two-dimensional target image corresponding to the target viewpoint (Lewis column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device) (position information is comprised in the image arrays in order to identify the image arrays); Ustarroz-Calonge abstract, generating at least a portion of a reconstructed image corresponding to the encoded image using the filtered alpha channel values and the color channel values).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the inventions of Lewis and Ustarroz-Calonge, to comprise the position information of different channels of the two-dimensional images in the encoded data, in order to better display the object.
Regarding claim 21. Lewis in view of Ustarroz-Calonge discloses The device according to claim 20, wherein the encoded data further comprises third attribute information of the two-dimensional images, and the third attribute information is used to label data frame types of different channels of the two-dimensional images in the encoded data (Lewis column 3 lines 6-9, A server may process the arrays of images and generate a 3D video of the object. The 3D video may allow a user to view a video of the object from multiple viewpoints; Ustarroz-Calonge [0051] The entropy-encoded coefficients, together with other information used to decode the block (which may include, for example, syntax elements such as used to indicate the type of prediction used, transform type, motion vectors, a quantizer value, or the like), are then output to the compressed bitstream 420); and
accordingly, the acquiring a color channel code corresponding to the target viewpoint from the encoded data according to the position information of the color channel, comprises:
according to the third attribute information, acquiring a data frame type of a color channel corresponding to the target viewpoint from the encoded data (Lewis column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device); Ustarroz-Calonge [0051] The entropy-encoded coefficients, together with other information used to decode the block (which may include, for example, syntax elements such as used to indicate the type of prediction used, transform type, motion vectors, a quantizer value, or the like), are then output to the compressed bitstream 420);
in response to the data frame type of the color channel corresponding to the target viewpoint being a non-key frame type, acquiring a key frame code and a non-key frame code of the color channel corresponding to the target viewpoint from the encoded data according to the position information of the color channel corresponding to the target viewpoint, and generating the color channel code corresponding to the target viewpoint according to the key frame code and non-key frame code of the color channel corresponding to the target viewpoint (Lewis column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device); Ustarroz-Calonge [0047] an intra/inter prediction stage 402; [0049] In the case of inter-prediction, a prediction block may be formed from samples in one or more previously constructed reference frames); and
in response to the data frame type of the color channel corresponding to the target viewpoint being a key frame type, acquiring the key frame code of the color channel corresponding to the target viewpoint from the encoded data according to the position information of the color channel corresponding to the target viewpoint, and determining the key frame code of the color channel corresponding to the target viewpoint as the color channel code corresponding to the target viewpoint (Lewis column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device); Ustarroz-Calonge [0047] an intra/inter prediction stage 402; [0049] In the case of intra-prediction, a prediction block may be formed from samples in the current frame that have been previously encoded and reconstructed).
The same motivations have been stated in claims 16 and 20.
Regarding claim 22. Lewis in view of Ustarroz-Calonge discloses The device according to claim 21, wherein the acquiring a transparency channel code corresponding to the target viewpoint from the encoded data according to the position information of the transparency channel, comprises:
according to the third attribute information, acquiring a data frame type of a transparency channel corresponding to the target viewpoint from the encoded data (Lewis column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device); Ustarroz-Calonge [0051] The entropy-encoded coefficients, together with other information used to decode the block (which may include, for example, syntax elements such as used to indicate the type of prediction used, transform type, motion vectors, a quantizer value, or the like), are then output to the compressed bitstream 420);
in response to the data frame type of the transparency channel corresponding to the target viewpoint being a non-key frame type, acquiring a key frame code and a non-key frame code of the transparency channel corresponding to the target viewpoint from the encoded data according to the position information of the transparency channel corresponding to the target viewpoint, and generating the transparency channel code corresponding to the target viewpoint according to the key frame code and non-key frame code of the transparency channel corresponding to the target viewpoint (Lewis column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device); Ustarroz-Calonge [0060] an alpha channel, where inter- or intra-prediction tools are used); and
in response to the data frame type of the transparency channel corresponding to the target viewpoint being a key frame type, acquiring the key frame code of the transparency channel corresponding to the target viewpoint from the encoded data according to the position information of the transparency channel corresponding to the target viewpoint, and determining the key frame code of the transparency channel corresponding to the target viewpoint as the transparency channel code corresponding to the target viewpoint (Lewis column 1 lines 46-53, The server may receive a request from a client to view the 3D video of the object from a new viewpoint. The server may identify a camera based on the new viewpoint. The server may identify an image array associated with the camera (e.g., identify the image array generated by the camera). The server may send the images from identified image array to a client (e.g., a client device and/or a media viewer operating on a client device); Ustarroz-Calonge [0060] an alpha channel, where inter- or intra-prediction tools are used).
The same motivations have been stated in claims 16 and 20.
Regarding claim 23. Lewis in view of Ustarroz-Calonge discloses The device according to claim 16, wherein the acquiring the two-dimensional image set generated by the three-dimensional object model at the plurality of different viewpoints, comprises:
performing compression encoding on the two-dimensional images corresponding to the plurality of different viewpoints to generate encoded data at the plurality of different viewpoints, and determining the encoded data at the plurality of different viewpoints as the two-dimensional image set (Lewis column 3 lines 6-9, A server may process the arrays of images and generate a 3D video of the object. The 3D video may allow a user to view a video of the object from multiple viewpoints; Ustarroz-Calonge [0027] the user interface 130 may be an audio-visual display device, and the computing device 100 may present audio, such as decoded audio, using the user interface 130 audio-visual display device, such as in conjunction with displaying video, such as decoded video; abstract, decoding, from multiple color channels of a bitstream, color channel values for an encoded image, decoding, from an alpha channel of the bitstream, alpha channel values for the encoded image; [0099] the transmitting computing and communication device 100A can be implemented on a server, and the receiving computing and communication device 100B can be implemented on a device separate from the server, the transmitting computing and communication device 100A can encode content using an encoder 400 into an encoded video signal and transmit the encoded video signal to the communications device, the communications device can then decode the encoded video signal using a decoder 500).
The same motivation has been stated in claim 16.
Regarding claim 1, the same analysis has been stated in claim 16.
Regarding claim 2, the same analysis has been stated in claim 18.
Regarding claim 4, the same analysis has been stated in claim 20.
Regarding claim 5, the same analysis has been stated in claim 21.
Regarding claim 6, the same analysis has been stated in claim 22.
Regarding claim 7. Ustarroz-Calonge discloses The image display method according to claim 2, wherein the encoded data comprises a protocol header, and the protocol header comprises one of a first attribute information, a second attribute information and a third attribute information, or any combination of the first attribute information, the second attribute information and the third attribute information ([0057] Using header information decoded from the compressed bitstream 420; [0036] The computing and communications system 200 shown includes computing and communication devices 100A, 100B, 100C, access points 210A, 210B, and a network 220; [0040] The network 220 can be any type of network configured to provide services. The network can use a communication protocol, such as the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), the Internet Protocol (IP), the Real-time Transport Protocol (RTP), the HyperText Transport Protocol (HTTP), or a combination thereof).
The same motivation has been stated in claim 16.
Regarding claim 8, the same analysis has been stated in claim 16.
Regarding claim 9, the same analysis has been stated in claim 23.
Regarding claim 10, the same analysis has been stated in claim 19.
Regarding claim 11, the same analysis has been stated in claim 20.
Regarding claim 12, the same analysis has been stated in claim 21.
Regarding claim 15, the same analysis has been stated in claim 16.
Claims 3, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Lewis et al. (US 9294757 B1) in view of Ustarroz-Calonge et al. (US 20220014792 A1) as applied above in claim 16, and further in view of Prins et al. (US 20200401362 A1).
Regarding claim 19. Prins discloses encoded data comprises first attribute information of two-dimensional images, size information in the first attribute information is used to label a display size of the two-dimensional images, and a target viewpoint is determined according to viewpoint information in the first attribute information ([0040] the sender system may generate metadata which is associated with the image data and provide the render metadata to the receiver system, the render metadata may be provided together with the image data to the receiver system, e.g., multiplexed in a same media stream; [0041]-[0043] the render metadata may indicate a size of the virtual canvas and a distance and/or orientation of the virtual canvas with respect to a viewpoint of a viewer in the virtual environment; [0179] the sender system to optimally generate the image data or optimally encode the image data).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the inventions of Lewis and Ustarroz-Calonge with the invention of Prins, to comprise the size information and the viewpoint information in the encoded data, in order to better display the object.
Regarding claim 3, the same analysis has been stated in claim 19.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIAOLAN XU whose telephone number is (571)270-7580. The examiner can normally be reached Mon. to Fri. 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, SATH V. PERUNGAVOOR can be reached at (571) 272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XIAOLAN XU/ Primary Examiner, Art Unit 2488