DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05 March 2026 has been entered.
Response to Arguments
Applicant's arguments filed 05 March 2026 have been fully considered but they are not persuasive.
Applicant argues “Hwang nowhere discloses what information determines the supported file format, nor is there any indication that such information corresponds to the ‘software used for generating a virtual viewpoint image’ … Hwang's disclosure is limited to hardware-related display capability” (Remarks, pgs. 7-8). The Examiner respectfully disagrees.
The claims recite “acquire information which indicates software used for generating a virtual viewpoint image in a different apparatus that is an output destination of the material data and which is for identifying a format processable by the different apparatus; convert, based on the software indicated by the information, a format of the acquired material data.” Hwang recites “when the format conversion unit receives 3D model information … it can convert the format of the received 3D model information according to the format of the 3D model information supported by the terminal device” (para. 79). This very clearly teaches the claimed “acquire information … which is for identifying a format processable by the different apparatus,” but Applicant argues that it does not teach the claimed “indicates software” part of the limitation. Hwang recites “render the received 3D model information to generate 2D image information … rendering can mean the process of automatically generating a photorealistic or non-photorealistic image from a three-dimensional model using a computer program” (paras. 110-111). This recitation from Hwang discloses that the terminal device uses software to generate images. Hwang further discloses “transmit whether the … terminal device is accessible, what functions it can provide, etc.” (para. 117) and “the terminal device can transmit various attribute information or status information of the terminal device, such as the format of the 3D model information supported by the terminal device” (para. 122). In total, Hwang teaches 1) a 3D model is rendered by the terminal using software; 2) only certain file types are “supported by” the terminal; and 3) the 3D model files are converted based on what file types are “supported by” the terminal, i.e. “what functions [the terminal] can provide.” When these are taken together, one can see that the Hwang teaches an indication of software on the terminal device, e.g. what file formats and types the software (and hardware) on the terminal device can support.
As another example, Hwang recites “converting a 3D model corresponding to the first model information into a 3D point cloud or a 3D mesh depending on the type of 3D model supported by the first terminal device” (para. 22). If both the hardware and software of the terminal device support a particular data format, then the terminal device would clearly transmit a status indicating support for that data format. However, if either the hardware or the software of the terminal device do not support a particular data format, then the terminal device would clearly transmit a status indicating that it does not support that data format. Thus, a transmission from the terminal device that it supports a particular data format (see para. 122), indicates that both the hardware and software support that data format. That is an indication of the software of the terminal device. The same logic used regarding the support of 3D meshes vs 3D point clouds can be applied to the support of different file formats described in para. 80.
In Applicant’s Remarks, Applicant cites paras. 78 and 80 to argue that Hwang does not convert a format of data based on an indication of software. But Applicant omits para. 79, which provides strong evidence in favor of Hwang teaching an indication of converting a data format based on what formats the terminal support: “when the format conversion unit receives 3D model information … it can convert the format of the received 3D model information according to the format of the 3D model information supported by the terminal device” (para. 79). Again, if a terminal device supports a certain model format, this indicates that the software of the terminal device supports that model format. This teaches the claimed “acquire information which indicates software used for generating a virtual viewpoint image in a different apparatus.”
Applicant argues “in the Advisory Action, the rationale merely describes the case where the terminal device supports the particular file format device, and another case where the software running on the terminal device does not support the file format is not all described in Hwang but is crucial to the claimed invention” (Remarks, pg. 9). The Examiner respectfully disagrees. Para. 79 of Hwang, cited above, very clearly teaches converting the file format to a format that is supported by the terminal device. This would not be necessary, and in fact would reduce performance, if the terminal device already supported the original format.
In summary, one can see from para. 111 of Hwang that the terminal device comprises software used for rendering 3D data. Further, many portions of Hwang teach that the 3D data is converted to a format that is supported by the terminal device. If the software of the terminal device does not support a particular 3D data format, then this format would clearly not be supported by the terminal device itself. Therefore, if the terminal device transmits a status that indicates that it supports a certain format, then this indicates that the software of the terminal device also supports that format.
Any remaining arguments are considered moot based on the foregoing.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 3, 4, 6, 8-10, 12-20, 24, and 25 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 6, 7, 9, 19-21, 23, 12-18, 25, and 27 of U.S. Patent 12,400,392 in view of Hwang et al. (KR 10-2020-0072321; hereinafter “Hwang”; machine translation used for citations). The patented claims teach each limitation (or a trivial variation) of the current claims, except for the language related to “converting” between formats and “information indicator software” (see claim limitation mapping below), which is rendered obvious by Hwang.
The following table illustrates a mapping of the conflicting claims:
Current Application
1
3, 4, 6
8-10, 12
13-20
24
25
U.S. Patent
1
6, 7, 9
19-21, 23
12-18
25
27
The following table illustrates a sample mapping of the limitations of claim 1 of the present application when compared against the limitations of claim 1 of the U.S. Patent:
Current Application
U.S. Patent
1. An information processing apparatus comprising: one or more memories storing instructions; and one or more processors executing the instructions to:
1. An information processing apparatus comprising: one or more memories storing instructions; and one or more processors executing the instructions to:
acquire material data that is to be used for generation of a virtual viewpoint image that is based on a plurality of captured images obtained by a plurality of imaging apparatuses performing image capturing of a subject,
acquire a plurality of material data for use in generation of a virtual viewpoint image based on a plurality of images captured with a plurality of imaging apparatuses by imaging an object,
and is material data represented by … the first format … a second format;
the plurality of material data including first material data and second material data different from the first material data;
and output the material data … [in the] second format, to the different apparatus
and output, to another apparatus ... material data that is selected … from among the acquired plurality of material data
The patented claims do not recite acquire information which indicates software used for generating a virtual viewpoint image in a different apparatus that is an output destination of the material data and which is for identifying a format processable by the different apparatus. The copending claims do not recite convert, based on the software indicated by the information, a format of the acquired material data, from the first format into a second format processable by the different apparatus or outputting the material data converted into the converted second format. In other words, the patented claims recite first and second material data, but do not disclose that the second material data is generated by converting the first material data.
In the same art of transmitting 3D data for display, Hwang teaches acquire information which indicates software used for generating a virtual viewpoint image in a different apparatus that is an output destination of the material data and which is for identifying a format processable by the different apparatus (“the format of the 3D model information supported by the terminal device … the format of 3D model information can mean various formats such as OBJ, MTL, STL, PLY, 3DS, COLLADA, XML, VRML, X3D, FBX, U3D, and O3D that express 3D model information,” paras. 79-81; different software packages support different formats, therefore an indication of a supported format is an indication of software that supports that format), and convert, based on the software indicated by the information, a format of the acquired material data, from the first format into a second format processable by the different apparatus and outputting the material data converted into the converted second format (“convert the format of the first model information according to the format of the three-dimensional model information supported by the first terminal device,” para. 10).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the techniques of Hwang to the patented claims. The motivation would have been to “effectively convert the quality of a 3D model according to the performance of a terminal device” (Hwang, para. 86).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4, 7-10, 13-22, 24, and 25 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Hwang et al. (KR 10-2020-0072321; hereinafter “Hwang”; machine translation used for citations).
Regarding claim 1, Hwang discloses An information processing apparatus comprising: one or more memories storing instructions; and one or more processors executing the instructions (“a holoportation service that supports various types of terminal devices,” para. 1) to: acquire material data that is to be used for generation of a virtual viewpoint image (“a first sensor unit that senses a first object; a first restoration server that restores a three-dimensional model of the first object through the sensing to generate first model information,” para. 8) that is based on a plurality of captured images obtained by a plurality of imaging apparatuses performing image capturing of a subject (“The sensor unit may be composed of sensors such as multiple cameras,” para. 42), and is material data represented by a first format (“first model information,” para. 8), and acquire information which indicates software used for generating a virtual viewpoint image in a different apparatus that is an output destination of the material data (“the format of the 3D model information supported by the terminal device … the format of 3D model information can mean various formats such as OBJ, MTL, STL, PLY, 3DS, COLLADA, XML, VRML, X3D, FBX, U3D, and O3D that express 3D model information,” paras. 79-81; “rendering … using a computer program,” para. 111; different software packages support different formats, therefore an indication of a supported format is an indication of software that supports that format) and which is for identifying a format processable by the different apparatus (“receive the format of 3D model information supported by each terminal device,” para. 78); convert, based on the software indicated by the information, a format of the acquired material data, from the first format into a second format processable by the different apparatus (“converts the first model information,” para. 8; “convert the format of the first model information according to the format of the three-dimensional model information supported by the first terminal device,” para. 10); and output the material data converted into the converted second format, to the different apparatus (“transmits the converted first model information; and a first terminal device that receives the converted first model information,” para. 8).
Regarding claim 2, Hwang discloses wherein a format of material data to be converted is a format for identifying a kind of material data (“convert a 3D model corresponding to the first model information into a 3D point cloud or a 3D mesh depending on the type of 3D model supported by the first terminal device,” para. 13).
Regarding claim 3, Hwang discloses wherein the information includes information regarding a type of the different apparatus (“the terminal device management unit can receive the identifier or IP address of the terminal device and information about the performance of the terminal device from each terminal device,” para. 106).
Regarding claim 4, Hwang discloses wherein the information includes information regarding processing capability of the different apparatus (“extract information on the performance of the terminal device,” para. 106).
Regarding claim 7, Hwang discloses generate a virtual viewpoint image represented by a third format, based on the acquired material data, and convert a format of the generated virtual viewpoint image, from the third format into a fourth format (“convert the format of the first model information or the second model information, or convert the quality, size, or type of a three-dimensional model corresponding to the first model information or the second model information,” para. 77).
Regarding claim 8, it is rejected using the same citations and rationales described in the rejection of claim 1, with the additional limitation generate a virtual viewpoint image based on the acquired material data (“generate 2D image information,” Hwang, para. 110; “a two-way holoportation providing system according to one embodiment is configured to include a first/second sensor unit, a first/second restoration server, a first/second relay server, and a first/second terminal device,” Hwang, para. 33).
Regarding claims 9 and 10, they are rejected using the same citations and rationales described in the rejections of claims 3 and 4, respectively.
Regarding claim 13, Hwang discloses wherein material data represented by the second format is data including point cloud data indicating a three-dimensional shape of the subject (“convert a 3D model corresponding to the first model information into a 3D point cloud,” para. 13), and texture data indicating a color of the subject (“3D model information … may include geometric information, color information,” para. 47).
Regarding claim 14, Hwang discloses wherein material data represented by the second format is data including point cloud data indicating a three-dimensional shape of the subject (“convert a 3D model corresponding to the first model information into a 3D point cloud,” para. 13) and a color of the subject (“3D model information … may include geometric information, color information,” para. 47).
Regarding claim 15, Hwang discloses wherein material data represented by the second format is data including mesh data indicating a three-dimensional shape of the subject (“convert a 3D model corresponding to the first model information into … a 3D mesh,” para. 13) and a color of the subject (“3D model information … may include geometric information, color information,” para. 47).
Regarding claim 16, Hwang discloses wherein material data represented by the second format is data including mesh data indicating a three-dimensional shape of the subject (“convert a 3D model corresponding to the first model information into … a 3D mesh,” para. 13), and texture data indicating a color of the subject (“3D model information … may include geometric information, color information,” para. 47).
Regarding claim 17, Hwang discloses wherein material data represented by the first format is data including point cloud data indicating a three-dimensional shape of the subject (“The first model information or the second model information may correspond to a 3D point cloud or mesh information,” para. 46), and texture data indicating a color of the subject (“3D model information … may include geometric information, color information,” para. 47).
Regarding claim 18, Hwang discloses wherein material data represented by the first format is data including point cloud data indicating a three-dimensional shape of the subject (“The first model information or the second model information may correspond to a 3D point cloud or mesh information,” para. 46) and a color of the subject (“3D model information … may include geometric information, color information,” para. 47).
Regarding claim 19, Hwang discloses wherein material data represented by the first format is data including mesh data indicating a three-dimensional shape of the subject (“The first model information or the second model information may correspond to a 3D point cloud or mesh information,” para. 46), and texture data indicating a color of the subject (“3D model information … may include geometric information, color information,” para. 47).
Regarding claim 20, Hwang discloses wherein material data represented by the first format is data including mesh data indicating a three-dimensional shape of the subject (“The first model information or the second model information may correspond to a 3D point cloud or mesh information,” para. 46) and a color of the subject (“3D model information … may include geometric information, color information,” para. 47).
Regarding claim 21, Hwang discloses wherein a format of the material data is a format regarding information for identifying a data amount of material data (“the 3D model quality conversion unit can adjust or improve the image quality of AR/VR, etc. displayed on the terminal device by proportionally reducing the number of vertices of the 3D model according to the performance of the terminal device,” para. 84).
Regarding claim 22, Hwang discloses wherein the information for identifying a data amount of material data includes information regarding a resolution of the material data (“the 3D model quality conversion unit can adjust or improve the image quality of AR/VR, etc. displayed on the terminal device by proportionally reducing the number of vertices of the 3D model according to the performance of the terminal device,” para. 84; a number of vertices used in a model can be considered a resolution of the 3D model).
Regarding claims 24 and 25, they are rejected using the same citations and rationales described in the rejection of claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Hwang in view of Liang et al. (US 11,308,687; hereinafter “Liang).
Regarding claim 6, Hwang does not specifically recite wherein the information includes information regarding a format of a virtual viewpoint image displayable by the different apparatus.
In the same art of transmitting 3D data for display, Liang teaches wherein the information includes information regarding a format of a virtual viewpoint image displayable by the different apparatus (“if the client device includes the ability to display shadows based on normals that can be transmitted to the device, then the type of data transmitted from the server can include those normals for rendering on the client device. However, if such a capability does not exist on the client device, then the server can generate shadow images and transmit the images or frames to the client device,” col. 10, lines 45-55).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Liang to Hwang. The motivation would have been to improve efficiency while maintaining image quality.
Regarding claim 12, it is rejected using the same citations and rationales described in the rejection of claim 6.
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Hwang in view of Xie et al. (US 2019/0378341; hereinafter “Xie”).
Regarding claim 23, Hwang does not disclose wherein the information for identifying a data amount of material data includes information regarding a frame rate of the material data.
In the same art of transmitting 3D data for display, Xie teaches wherein the information for identifying a data amount of material data includes information regarding a frame rate of the material data (“The device characteristics may be received from a client device, such as in association with a request for a 3D model of an item … e.g., a threshold rendering frame rate or resolution,” para. 42).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Xie to Hwang. The motivation would have been to “automatically generate optimized sampled 3D models” (Xie, para. 20).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ryan McCulley whose telephone number is (571)270-3754. The examiner can normally be reached Monday through Friday, 8:00am - 4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN MCCULLEY/Primary Examiner, Art Unit 2611
Prosecution Insights