DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “110” has been used to designate both segmentation and materials, and “115” has been used to designate both segmented 3D surface and rendering engine. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities: Fig. 2D (0010) and Fig. C (0012).
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 11, 13, 16, 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ford (US20180300936A1) in view of Sharp (US20250308163A1).
Regarding claim 1, Ford teaches a method, comprising:
receiving information about a three-dimensional (“3D”) surface; “3D modeling component 210 can identify the various aspects (interpreted as information) based on input received from capture component 230” (Ford, 0071).
determining one or more segments of the 3D surface based on the information about the 3D surface; “the 3D modeling component 110 can identify a portion of the captured 3D data (e.g., a section of a mesh)” (Ford, 0052).
Ford also teaches:
“a 3D model can be associated with a set of textures (interpreted as segments)” (Ford, 0087).
“a segmentation component can dynamically segment textures (interpreted as material type) associated with a 3D model” (Ford, 0035).
“a segmentation component can dynamically segment textures associated with a 3D model to facilitate efficient allocation of a texture memory. Segmenting the textures can comprise determining a set of segments from captured imagery data, rearranging (e.g., combining, dividing, removing, or otherwise altering) segments of an already generated set of segments” (Ford, 0035).
Ford doesn’t explicitly teach providing, to a large language model (“LLM”), a prompt comprising the one or more segments, but Sharp teaches:
providing, to a large language model (“LLM”), a prompt comprising the one or more segments; “the processor is comprised in at least one of a system that can include one or more language models, such as one or more large language models (LLMs)” (Sharp, 0099).
receiving, from the LLM, a material type associated with each of the one or more segments; “Parameterizations can store appearance data such as colors, materials, etc., for a 3D mesh in an accompanying image file called a texture” (Sharp, 0026).
determining, using a materials database, a material for each of the one or more segments based on the associated material type; “Parameterizations can store appearance data such as colors, materials, etc., for a 3D mesh in an accompanying image file called a texture” (Sharp, 0026).
and rendering an image of the 3D surface, wherein the rendered image shows each segment of the one or more segments with the determined material applied. “Output the data (e.g., as an image)” (Sharp, 0069).
Sharp also teaches:
“The input is a mesh. Parameterizations can store appearance data such as colors, materials, etc., for a 3D mesh in an accompanying image file called a texture” (Sharp, 0026).
Ford and Sharp are combinable because they are in the same field of endeavor of 3D modeling for computer graphics. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the application to combine system of Ford and LLM of Sharp in order to minimize overlap and distortion require significant manual input, and significant computational resources to perform projections that do not sufficiently mitigate distortion and overlap (Sharp, 0002).
Regarding claim 2, Ford teaches the method of claim 1, wherein the material type associated with each of the one or more segments further includes:
a first text that identifies the 3D surface; and for each of the one or more segments: “At least in part on the first area (interpreted as first text) of texture data (interpreted as material type) and the second area (interpreted as second text) of the texture data. In an aspect, a system can assign or map the first area and second area to areas of a 3D mesh” (interpreted as 3D surface) (Ford, 0122).
a second text that identifies the segment; “At least in part on the first area (interpreted as first text) of texture data (interpreted as material type) and the second area (interpreted as second text) of the texture data. In an aspect, a system can assign or map the first area and second area to areas of a 3D mesh” (interpreted as 3D surface) and “identifying patterns within a texture and allocating texture memory to texture regions (e.g., areas, portions, segments, etc.)” (Ford, 0122).
and a color associated with the segment. “Texture can be associated with portions of the mesh. The texture can comprise image data (e.g., pixels) describing texture, color, intensity, and the like” (Ford, 0025).
Regarding claim 3, Ford teaches, the method of claim 1, wherein the information about the 3D surface comprises information about one or more base textures and a specification of a 3D mesh. “Determine to utilize computer generated shading or texturing techniques (interpreted as base textures), such as vertex coloring, to texture a particular region (interpreted as a specification) of a 3D mesh” (Ford, 0058).
Regarding claim 4, Ford teaches, the method of claim 3, wherein:
the information about the 3D surface further comprises segmentation information about the 3D mesh; “segmenting textures can also include configuring or segmenting a 3D mesh” (Ford, 0035).
and determining the one or more segments of the 3D surface comprises accessing the segmentation information from the information about the 3D surface. “Identify a portion of the captured 3D data (e.g., a section of a mesh) that corresponds to a particular flat surface and/or associate the portion of the captured 3D data (e.g., a section of the mesh) with the particular flat surface” (Ford, 0052).
Regarding claim 5, Ford teaches, the method of claim 3, wherein determining the one or more segments of the 3D surface comprises using a mesh segmentation algorithm to segment the 3D mesh. “Method 700 can provide for judiciously segmenting texture data and altering the texture data. For example, a system can receive a 3D model or 3D imagery data and segment it into a 3D mesh and texture areas” (Ford, 0014).
Regarding claim 11, claim 11 is similar in scope of claim 1. Ford teaches a system comprising:
one or more processors;
and one or more computer-readable storage media storing instructions which, when executed by the one or more processors, cause the one or more processors to perform operations including: “Computing devices typically include a variety of media, which can include computer-readable storage media or communications media” (Ford, 0154).
receiving information about a three-dimensional 3D surface, comprising information about one or more base textures and a specification of a 3D mesh; “3D modeling component 210 can identify the various aspects (interpreted as information) based on input received from capture component 230” (Ford, 0071) and “determine to utilize computer generated shading or texturing techniques (interpreted as base textures), such as vertex coloring, to texture a particular region (interpreted as a specification) of a 3D mesh” (Ford, 0058).
determining one or more segments of the 3D surface based on the information about the 3D surface; “the 3D modeling component 110 can identify a portion of the captured 3D data (e.g., a section of a mesh)” (Ford, 0052).
Ford also teaches:
“a segmentation component can dynamically segment textures (interpreted as material type) associated with a 3D model” (Ford, 0035).
“a segmentation component can dynamically segment textures associated with a 3D model to facilitate efficient allocation of a texture memory. Segmenting the textures can comprise determining a set of segments from captured imagery data, rearranging (e.g., combining, dividing, removing, or otherwise altering) segments of an already generated set of segments” (Ford, 0035).
Ford doesn’t explicitly teach determining, using an LLM, a material type associated with each of the one or more segments, but Sharp teaches:
determining, using an LLM, a material type associated with each of the one or more segments; “the processor is comprised in at least one of a system that can include one or more language models, such as one or more large language models (LLMs)” (Sharp, 0099).
determining, using a materials database, a material for each of the one or more segments based on the associated material type; “Parameterizations can store appearance data such as colors, materials, etc., for a 3D mesh in an accompanying image file called a texture” (Sharp, 0026).
and rendering an image of the 3D surface, wherein the rendered image shows each segment of the one or more segments with the determined material applied. “Output the data (e.g., as an image)” (Sharp, 0069).
Sharp also teaches:
“The input is a mesh. Parameterizations can store appearance data such as colors, materials, etc., for a 3D mesh in an accompanying image file called a texture” (Sharp, 0026).
Ford and Sharp are combinable because they are in the same field of endeavor of 3D modeling for computer graphics. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the application to combine system of Ford and LLM of Sharp in order to minimize overlap and distortion require significant manual input, and significant computational resources to perform projections that do not sufficiently mitigate distortion and overlap (Sharp, 0002).
Regarding claim 13, the claim is directed to a method claim with the similar limitations as claim 2. As such, claim 13 is rejected on the same grounds.
Regarding claim 16, the claim is directed to a method claim with the similar limitations as claim 1 and 11. As such, claim 16 is rejected on the same grounds.
Regarding claim 17, the claim is directed to a method claim with the similar limitations as claim 5. As such, claim 17 is rejected on the same grounds.
Claim(s) 6, 8, 12, 14, 18, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ford (US20180300936A1) and Sharp (US20250308163A1) as applied to claim 3, 11, 16 above and further in view of Zhou (US20250232471A1).
Regarding claim 6, Ford as modified teaches, the method of claim 3, wherein providing, to the LLM, the prompt comprising the one or more segments, and rendering a second image of the 3D surface based on the specification of the 3D mesh and the one or more segments; “determine to utilize computer generated shading or texturing techniques (interpreted as base textures), such as vertex coloring, to texture a particular region (interpreted as a specification) of a 3D mesh” (Ford, 0058).
Ford doesn’t teach generating the prompt for the LLM including instructions to identify the material type for each of the one or more segments, but Zhou teaches:
generating the prompt for the LLM including instructions to identify the material type for each of the one or more segments; “For embodiments that incorporate one or more language models—that is, one or more LLMs, one or more VLMs, or a combination of LLMs and VLMs, the language model(s) may receive an input (e.g., a prompt, a request, a query, etc.)” and “an input generator (e.g., a prompt generator), which may be driven or otherwise guided by one or more AI and/or ML systems, may be used to generate this input based on an initial input received from a user, a device, a proxy, and/or the like” (Zhou, 0024).
and providing, to the LLM, the second image of the 3D surface and the prompt. “Process the input image and features” (Zhou, 0049).
Ford, Sharp, and Zhou are combinable because they are in the same field of endeavor of 3D modeling for computer graphics. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the application to combine system of Ford and LLM of Sharp as modified for second image of Zhou in order to bring realism to improve efficiency by encoding a scene using a compact neural network, such learned scene representations lack interpretability needed for accurate visual localization (Zhou, 0002).
Regarding claim 8, Ford as modified teaches, the method of claim 3, wherein determining, using the materials database, the material for each of the one or more segments based on the associated material type comprises:
for each of the one or more segments:
rendering a second image of the 3D surface based on the specification of the 3D mesh, the one or more segments, and information about a texture of the one or more base textures; “the 3D modeling component 110 can identify a portion of the captured 3D data (e.g., a section of a mesh)” (Ford, 0052).
Ford also teaches:
“a segmentation component can dynamically segment textures associated with a 3D model to facilitate efficient allocation of a texture memory. Segmenting the textures can comprise determining a set of segments from captured imagery data, rearranging (e.g., combining, dividing, removing, or otherwise altering) segments of an already generated set of segments” (Ford, 0035).
and determining a sub-image of the second image, the second image including the texture; “a subset of the training images” (Zhou, 0044).
and wherein determining, using the materials database, the material for each of the one or more segments based on the associated material type is based on the sub-images. “Parameterizations can store appearance data such as colors, materials, etc., for a 3D mesh in an accompanying image file called a texture” (Sharp, 0026).
Regarding claim 12, the claim is directed to a method claim with the similar limitations as claim 1 and 6. As such, claim 12 is rejected on the same grounds.
Regarding claim 14, the claim is directed to a method claim with the similar limitations as claim 8. As such, claim 14 is rejected on the same grounds.
Regarding claim 18, the claim is directed to a method claim with the similar limitations as claim 6. As such, claim 18 is rejected on the same grounds.
Regarding claim 19, the claim is directed to a method claim with the similar limitations as claim 14. As such, claim 19 is rejected on the same grounds.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ford (US20180300936A1), Sharp (US20250308163A1), and Zhou as applied to claim 6, and further in view of Fradet (US 20170337741A1).
Regarding claim 7, Ford as modified teach the method of claim 6, but does not teach further comprise rendering a third image of the 3D surface based on the information about the one or more base textures and wherein providing, to the LLM, further includes the third image, but Fradet teaches “Rendering, in the at least one image, at least one first part of the transformed virtual object by using texture information” (0010).
Ford as modified and Fradet are combinable because they are in the same field of endeavor of 3D modeling for computer graphics. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the application to combine system of Ford and LLM of Sharp as modified for third image of Fradet in order to bring realism to the scene (Fradet, abstract).
Claim(s) 9 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ford (US20180300936A1, Sharp (US20250308163A1), and Zhou as applied to claim 8 and 14 above, and further in view of Imber (US20140354645A1).
Regarding claim 9, Ford, Sharp, and Zhou teach the method of claim 8, but do not teach wherein determining, using the materials database, the material for each of the one or more segments based on the associated material type is further based on a feature similarity and a color histogram associated with each of the sub-images, but Imber teaches “The histogram was found to be a good indicator of colour albedo of a material” (Imber, 0036).
Ford also teaches:
“The system can determine segment sizes and positions based on identified similarity of image data or visual quality of image data, or other characteristics” (Ford, 0115).
Ford, Sharp, Zhou, and Imber are combinable because they are in the same field of endeavor of 3D modeling for computer graphics. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the application to combine system of Ford and LLM of Sharp as modified for color texture of Imber and second image of Zhou in order to increase the plausibility of the relighting of the scene (Imber, 0022).
Regarding claim 15, the method of claim 14, the claim is directed to a method claim with the similar limitations as claim 9. As such, claim 15 is rejected on the same grounds.
Claim(s) 10 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ford (US20180300936A1) and Sharp (US20250308163A1) as applied to claim 3 and 16 above, and further in view of Lieckfeldt (US8705843B2) and Brenner (US20220046221A1).
Regarding claim 10, Ford in view of Sharp teaches, the method of claim 3, wherein rendering the image of the 3D surface comprises:
for each of the one or more segments:
determining a subset of the 3D mesh corresponding to the segment; “the 3D modeling component 110 can identify a portion of the captured 3D data (e.g., a section of a mesh) that corresponds to a particular flat surface and/or associate the portion of the captured 3D data (e.g., a section of the mesh) with the particular flat surface” (Ford, 0052).
Neither Ford nor Sharp teaches scaling the determined material based on a size of the segment, normalizing a height of the determined material based on the one or more determined materials, and applying the normalized determined material to the subset of the 3D.
However, Lieckfeldt teaches:
scaling the determined material based on a size of the segment; “scalable attributes of the raw texture element, particularly the height and width and/or picture element numbers of the raw texture element, are rescaled to a normalized reference quantity” (Lieckfeldt, 38).
normalizing a height of the determined material based on the one or more determined materials; “scalable attributes of the raw texture element, particularly the height and width and/or picture element numbers of the raw texture element, are rescaled to a normalized reference quantity” (Lieckfeldt, 38).
Ford, Sharp, and Lieckfeldt are combinable because they are in the same field of endeavor of 3D modeling for computer graphics. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the application to combine system of Ford and LLM of Sharp as modified for size scaling of Lieckfeldt in order to reduces the library elements which might be necessary for the exchange of a raw texture element (Lieckfeldt, 38).
Furthermore, Brenner teaches:
and applying the normalized determined material to the subset of the 3D mesh “applying the subset of the panoramic images as textures on the 3D mesh” (Brenner, 0007).
Ford, Sharp, Lieckfeldt, and Brenner are combinable because they are in the same field of endeavor of 3D modeling for computer graphics. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the application to combine system of Ford and LLM of Sharp as modified for size scaling of Lieckfeldt and material of Brenner in order to allows the size and shape of the environment to be used in many processes (Brenner, 0003).
Regarding claim 20, the method of claim 16, the claim is directed to a method claim with the similar limitations as claim 10. As such, claim 20 is rejected on the same grounds.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIANA SZE whose telephone number is (571)272-9916. The examiner can normally be reached Monday-Thursday 6am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.S./Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights