DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
2. Receipt is acknowledged of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file.
Information Disclosure Statement
3. The information disclosure statement (IDS) submitted on 08/20/2024. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
7. Claim(s) 1-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cheshire et al. (US 2022/0035970 A1) in view of Hosenpud et al. (US 2018/0205939 A1).
8. With reference to claim 1, Cheshire teaches An image display system (“a 3D modeling system configured to automatically generate photorealistic, virtual 3D package and product models from 3D and 2D imaging assets. The 3D modeling system may include one or more processors and an automatic imaging asset assembly script configured to execute on the one or more processors. … The one or more processors of the 3D modeling system may be configured to render, via a graphical display, the virtual 3D model as a photorealistic image representing the real-world product or product package.” [0009]) Cheshire also teaches customer image storage means that stores two-dimensional image data provided by a customer; container data storage means that stores three-dimensional shape data of a container provided by a vendor; (“As described herein a “memory” may refer to either memory 106 and/or database 105. Such memory may be configured to store 2D imaging assets and 3D imaging assets accessible by processor(s) 104, scripts, application, or other software, … database 105 may be a product lifecycle management (PLM) database or system. Generally, a PLM database or system is implemented as an information management system that can integrate data, processes, and other business systems within an enterprise or platform, such as the platform depicted for 3D modeling system 100. A PLM database or system generally includes software for managing information (e.g., 3D imaging assets and 2D imaging assets) throughout an entire lifecycle of a product/package in an efficient and cost-effectivities manner. The lifecycle may include lifecycle stages from ideation, design and manufacture, through service and disposal. In some embodiments, database 105 may store digital PLM objects (e.g., digital 3D imaging assets and/or 2D imaging assets as described herein). Such digital objects or assets can represent a real-world physical parts, assemblies(s), or documents, customer requirements or supplier parts, a change process, and/or other data types relating to a lifecycle management and development of a product and/or package. For example, digital objects or assets can include computer-aided design (CAD) file(s) that depict or describe (e.g., via measurements, sizes, etc.) parts, components, or complete (or partially complete) models or designs of products and/or packages. Generally, non-CAD files can also be included database 105. Such non-CAD files can include text or data files describing or defining parts, components, and/or product or package specifications, vendor datasheets, or emails relating to a design.” [0042-0043] “processor(s) 104 may store virtual 3D model(s) in memory 106 and/or database 105 such that virtual 3D model(s) are accessible to an automatic imaging asset assembly script or a visualization editor.” [0045] “a predefined design shape corresponding to a real-world product or product package (e.g., a shampoo bottle with a label or package of toilet paper with a wrapper) is selected for submission and search of PLM database system 210. As shown in the embodiment of FIG. 2, a user may select the predefined design shape from a user interface 209. In other embodiments, one or more predefined design shape(s) may be loaded into a script for submission to the PLM database system 210 without user selection. Various types of shapes may be selected or used as the predefined design shape 3D.” [0058] “one or more digital surface finish artifacts of a virtual material library 520, as selected from the 2D imaging assets, may be applied (522) to a polygonal model. For example, as shown in FIG. 5A, the physical materials/CMF 318 are applied (522), by processor(s) 104, to high resolution polygonal model 510H generate a virtual 3D model 810M of the real-world product or product package (e.g., shampoo bottle). Physical materials/CMF 318 may include product surface textures, print finishes, colors, appearances, and finishes (e.g., smooth, shiny, water, wood, metal, grain, etc.). Such Physical materials/CMF 318 may be stored in CAD XML file 350 for access by polygonal model generation script 502, processor(s) 104, and/or a 3D software of a visualization editors. The physical materials/CMF 318 values are applied to high resolution polygonal model 510H by adding the surface textures, print finishes, colors, appearances, etc. to the surface or other area of high resolution polygonal model 510H.” [0086]) Cheshire further teaches the two-dimensional image data includes first two-dimensional image data rendered in a single color, and second two-dimensional image data rendered in a plurality of colors, (“Physical materials/CMF 318 may include product surface textures, print finishes, colors, appearances, and finishes (e.g., smooth, shiny, water, wood, metal, grain, etc.). Such physical materials/CMF 318 may be stored in CAD XML file 350 for access by polygonal model generation script 502, processor(s) 104, and/or a 3D software of a visualization editors.” [0096] “artwork 704 may be split into masks or polygons in order for processor(s) 104 to apply different finishes or colors swatches to match materials. If masks are used, processor(s) 104 may select corresponding virtual materials based artwork 704 that is applied to high resolution polygonal model 510H.” [0100] “processor(s) 104 accesses back plates or background images 714 and/or lighting effects 716 for updating and/or rendering color or chromatic properties of high resolution polygonal model 510H. For example, in some embodiments a COLORWAY element file, with meta-tags identifying surfaces or areas of high resolution polygonal model 510H for coloring, may loaded or populated. At block 750, processor(s) 104 apply the colors of a color sheet 744 (e.g., COLORWAY sheet) to the high resolution polygonal model 510H. Color sheet 744 may be loaded or accessed by processors 104(s), from memorie(s) 106, and may define color(s) to be applied, e.g., virtually painted, on surfaces or areas (e.g., polygons or pixels) of high resolution polygonal model 510H. For example, color may be applied to artwork 752 (which may include artwork 704, e.g., drawings, pictures, etc., as described for FIG. 7A). In addition, color palettes 754, which may include different color sets, such as pre-defined color sets of matching or complementary colors for application to products and/or packages, may be loaded, by processor(s) 104, and applied, or used to paint, high resolution polygonal model 510H. In some embodiments, a user may select colors or color palettes from a visualization editor for application to high resolution polygonal model 510H, including to the materials, finishes, artwork, or other surface changes applied to high resolution polygonal model 510H as described herein.” [0104-0105])
PNG
media_image1.png
404
608
media_image1.png
Greyscale
Cheshire does not explicitly teach stereoscopic image creation means that creates a stereoscopic image by superimposing the two-dimensional image data on the three-dimensional shape data; and output means that outputs the stereoscopic image to a display device, and the stereoscopic image creation means includes (A) first stereoscopic image creation means that creates a first stereoscopic image by superimposing the first two-dimensional image data on the three-dimensional shape data, (B) second stereoscopic image creation means that creates a second stereoscopic image by superimposing the second two-dimensional image data on the three-dimensional shape data, and (C) third stereoscopic image creation means that creates a third stereoscopic image by superimposing the second two-dimensional image data on the first stereoscopic image. This is what Hosenpud teaches (“Either or both of the displays 150A and 150B may present (display) stereoscopic images for viewing by the user. By presenting stereoscopic images, the display(s) 150 may present a 3D scene for the user. This 3D scene may be considered or referred to as an illusion or simulated 3D because the actual provided images are 2D, but the scene is conveyed in 3D via the user's interpretation of the provided images via stereoscopic effects.” [0074] “webpages have predominantly revolved around presenting two-dimensional (2D) content, e.g., text, images, videos, and so forth, to end users. However, the introduction of the hypertext markup language (HTML) canvas element and Web Graphics Library (WebGL) has enabled webpages to present three-dimensional (3D) content as well.” [0105] “one or more layout properties of a WebGL canvas may be specified to allow for integration of 3D content with 2D content within a webpage. … the WebGL canvas may be specified with a transparent background (e.g., to allow the WebGL canvas to be overlaid on top of or in front of rendered 2D content and still retain visualization of the rendered 2D content). In some embodiments, the WebGL canvas may be specified as full screen such that a virtual beam associated with the user input device may be rendered to appear that the virtual beam is interacting with the web browser's user interface and scrollbars (anything outside of the browser's client viewport area). FIG. 7 illustrates a block diagram of an example of a method for rendering a webpage that includes both 2D and 3D content, according to some embodiments. The method shown in FIG. 7 may be used in conjunction with any of the systems or devices shown in the above Figures, among other devices. In various embodiments, some of the method elements shown may be performed concurrently, in a different order than shown, or may be omitted. … a webpage including one or more layers may be rendered on a surface of a display device, such as one of the display devices described above. The layers may include a first layer associated with 3D content, e.g., a WebGL canvas, and a second layer associated with 2D content. In some embodiments, a single WebGL canvas may be configured to support multiple instances of 3D content. … the rendering may be a 3D stereoscopic rendering.” [0111-0113]) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Hosenpud into Cheshire, in order to interact with 2D content.
9. With reference to claim 2, Cheshire does not explicitly teach the output means outputs, to the display device, a selection button that allows selection of an image to be displayed on the display device from among the first to third stereoscopic images. This is what Hosenpud teaches (“Either or both of the displays 150A and 150B may present (display) stereoscopic images for viewing by the user. By presenting stereoscopic images, the display(s) 150 may present a 3D scene for the user. This 3D scene may be considered or referred to as an illusion or simulated 3D because the actual provided images are 2D, but the scene is conveyed in 3D via the user's interpretation of the provided images via stereoscopic effects.” [0074] “user input device 600 may include buttons 604, 606, and 612. One of the buttons, such as button 606, may be depressed and held down to trigger the selection of an object within a 3D scene presented by any of systems 100, 500, and 550. … webpages have predominantly revolved around presenting two-dimensional (2D) content, e.g., text, images, videos, and so forth, to end users. However, the introduction of the hypertext markup language (HTML) canvas element and Web Graphics Library (WebGL) has enabled webpages to present three-dimensional (3D) content as well.” [0104-0105] “one or more layout properties of a WebGL canvas may be specified to allow for integration of 3D content with 2D content within a webpage. … the WebGL canvas may be specified with a transparent background (e.g., to allow the WebGL canvas to be overlaid on top of or in front of rendered 2D content and still retain visualization of the rendered 2D content). In some embodiments, the WebGL canvas may be specified as full screen such that a virtual beam associated with the user input device may be rendered to appear that the virtual beam is interacting with the web browser's user interface and scrollbars (anything outside of the browser's client viewport area). FIG. 7 illustrates a block diagram of an example of a method for rendering a webpage that includes both 2D and 3D content, according to some embodiments. The method shown in FIG. 7 may be used in conjunction with any of the systems or devices shown in the above Figures, among other devices. In various embodiments, some of the method elements shown may be performed concurrently, in a different order than shown, or may be omitted. … a webpage including one or more layers may be rendered on a surface of a display device, such as one of the display devices described above. The layers may include a first layer associated with 3D content, e.g., a WebGL canvas, and a second layer associated with 2D content. In some embodiments, a single WebGL canvas may be configured to support multiple instances of 3D content. … the rendering may be a 3D stereoscopic rendering.” [0111-0113]) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Hosenpud into Cheshire, in order to interact with 2D content.
10. With reference to claim 3, Cheshire teaches three-dimensional shape data of a selected container. (“a predefined design shape corresponding to a real-world product or product package (e.g., a shampoo bottle with a label or package of toilet paper with a wrapper) is selected for submission and search of PLM database system 210. As shown in the embodiment of FIG. 2, a user may select the predefined design shape from a user interface 209. In other embodiments, one or more predefined design shape(s) may be loaded into a script for submission to the PLM database system 210 without user selection. Various types of shapes may be selected or used as the predefined design shape 3D.” [0058] “one or more digital surface finish artifacts of a virtual material library 520, as selected from the 2D imaging assets, may be applied (522) to a polygonal model. For example, as shown in FIG. 5A, the physical materials/CMF 318 are applied (522), by processor(s) 104, to high resolution polygonal model 510H generate a virtual 3D model 810M of the real-world product or product package (e.g., shampoo bottle). Physical materials/CMF 318 may include product surface textures, print finishes, colors, appearances, and finishes (e.g., smooth, shiny, water, wood, metal, grain, etc.). Such Physical materials/CMF 318 may be stored in CAD XML file 350 for access by polygonal model generation script 502, processor(s) 104, and/or a 3D software of a visualization editors. The physical materials/CMF 318 values are applied to high resolution polygonal model 510H by adding the surface textures, print finishes, colors, appearances, etc. to the surface or other area of high resolution polygonal model 510H.” [0086])
Cheshire does not explicitly teach the output means is capable of displaying a stereoscopic image in which the two-dimensional image data is not superimposed on three-dimensional data. This is what Hosenpud teaches (“Either or both of the displays 150A and 150B may present (display) stereoscopic images for viewing by the user. By presenting stereoscopic images, the display(s) 150 may present a 3D scene for the user. This 3D scene may be considered or referred to as an illusion or simulated 3D because the actual provided images are 2D, but the scene is conveyed in 3D via the user's interpretation of the provided images via stereoscopic effects.” [0074] “webpages have predominantly revolved around presenting two-dimensional (2D) content, e.g., text, images, videos, and so forth, to end users. However, the introduction of the hypertext markup language (HTML) canvas element and Web Graphics Library (WebGL) has enabled webpages to present three-dimensional (3D) content as well.” [0105] “one or more layout properties of a WebGL canvas may be specified to allow for integration of 3D content with 2D content within a webpage. … the WebGL canvas may be specified with a transparent background (e.g., to allow the WebGL canvas to be overlaid on top of or in front of rendered 2D content and still retain visualization of the rendered 2D content). In some embodiments, the WebGL canvas may be specified as full screen such that a virtual beam associated with the user input device may be rendered to appear that the virtual beam is interacting with the web browser's user interface and scrollbars (anything outside of the browser's client viewport area). FIG. 7 illustrates a block diagram of an example of a method for rendering a webpage that includes both 2D and 3D content, according to some embodiments. The method shown in FIG. 7 may be used in conjunction with any of the systems or devices shown in the above Figures, among other devices. In various embodiments, some of the method elements shown may be performed concurrently, in a different order than shown, or may be omitted. … a webpage including one or more layers may be rendered on a surface of a display device, such as one of the display devices described above. The layers may include a first layer associated with 3D content, e.g., a WebGL canvas, and a second layer associated with 2D content. In some embodiments, a single WebGL canvas may be configured to support multiple instances of 3D content. … the rendering may be a 3D stereoscopic rendering.” [0111-0113]) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Hosenpud into Cheshire, in order to interact with 2D content.
11. With reference to claim 4, Cheshire teaches the container is a flexible packaging container. (“PLM objects or assets, and/or corresponding data records, such as those that may be stored in database 105, can contain properties regarding an object's or an asset's parameters or aspects of its design lifecycle. For example, PLM database or systems can generally store different classes of objects or assets (primarily parts (e.g., as CAD files), documents, and change forms) with distinct properties and behaviors. Such properties can include metrics or metadata such as part/document number, item category, revision, title, unit of measure, bill of materials, cost, mass, regulatory compliance details, file attachments, and other such information regarding product(s), and/or package(s) of a company. In addition, such PLM objects or assets may be linked, e.g., within database 105 (e.g., as a relational database), to other objects or assets within database 105 for the association of or otherwise generation or construction of a product structure. In this way, a PLM database can be flexibly used to identify objects and assets, create and define relationships among such objects and assets. Such flexibility provides a basis for the creation, customization, revision, and/or reuse of virtual models (e.g., virtual 3D models) as described herein, and also the 3D and 2D imaging assets on which they are based.” [0044] “virtual 3D model 810M is a high resolution polygonal model (e.g., high resolution polygonal model 510H) with virtual materials, finishes, artwork (e.g., artwork for or comprising labels, flexible wrappers, etc.), colors, and other surface elements or object, as described herein, applied such that virtual 3D model 810M renders, e.g., on a graphic display, as a photorealistic image representing a real-world product or product package (e.g., shampoo bottle).” [0106])
12. With reference to claim 5, Cheshire teaches gloss information imparting means that imparts, to the three-dimensional shape data, information indicating that the container has a glossy surface. (“a predefined design shape corresponding to a real-world product or product package (e.g., a shampoo bottle with a label or package of toilet paper with a wrapper) is selected for submission and search of PLM database system 210. As shown in the embodiment of FIG. 2, a user may select the predefined design shape from a user interface 209. In other embodiments, one or more predefined design shape(s) may be loaded into a script for submission to the PLM database system 210 without user selection. Various types of shapes may be selected or used as the predefined design shape 3D.” [0058] “one or more digital surface finish artifacts of a virtual material library 520, as selected from the 2D imaging assets, may be applied (522) to a polygonal model. For example, as shown in FIG. 5A, the physical materials/CMF 318 are applied (522), by processor(s) 104, to high resolution polygonal model 510H generate a virtual 3D model 810M of the real-world product or product package (e.g., shampoo bottle). Physical materials/CMF 318 may include product surface textures, print finishes, colors, appearances, and finishes (e.g., smooth, shiny, water, wood, metal, grain, etc.). Such Physical materials/CMF 318 may be stored in CAD XML file 350 for access by polygonal model generation script 502, processor(s) 104, and/or a 3D software of a visualization editors. The physical materials/CMF 318 values are applied to high resolution polygonal model 510H by adding the surface textures, print finishes, colors, appearances, etc. to the surface or other area of high resolution polygonal model 510H.” [0086])
13. With reference to claim 6, Cheshire teaches the first two-dimensional image data is rendered in a single white color. (“artwork 704 may be split into masks or polygons in order for processor(s) 104 to apply different finishes or colors swatches to match materials. If masks are used, processor(s) 104 may select corresponding virtual materials based artwork 704 that is applied to high resolution polygonal model 510H.” [0100] “processor(s) 104 accesses back plates or background images 714 and/or lighting effects 716 for updating and/or rendering color or chromatic properties of high resolution polygonal model 510H. For example, in some embodiments a COLORWAY element file, with meta-tags identifying surfaces or areas of high resolution polygonal model 510H for coloring, may loaded or populated. At block 750, processor(s) 104 apply the colors of a color sheet 744 (e.g., COLORWAY sheet) to the high resolution polygonal model 510H. Color sheet 744 may be loaded or accessed by processors 104(s), from memorie(s) 106, and may define color(s) to be applied, e.g., virtually painted, on surfaces or areas (e.g., polygons or pixels) of high resolution polygonal model 510H. For example, color may be applied to artwork 752 (which may include artwork 704, e.g., drawings, pictures, etc., as described for FIG. 7A). In addition, color palettes 754, which may include different color sets, such as pre-defined color sets of matching or complementary colors for application to products and/or packages, may be loaded, by processor(s) 104, and applied, or used to paint, high resolution polygonal model 510H. In some embodiments, a user may select colors or color palettes from a visualization editor for application to high resolution polygonal model 510H, including to the materials, finishes, artwork, or other surface changes applied to high resolution polygonal model 510H as described herein.” [0104-0105])
14. With reference to claim 7, Cheshire teaches the output means is capable of outputting, to the image, a mark indicating a position of a fastener of the container. (“CAD part 311 is a cap of a shampoo bottle; CAD part 312 is a body of the shampoo bottle; and CAD part 313 is a label of the shampoo bottle. Each of these components is shown as rendered as a virtual or 3D component. In addition, in various embodiments, each of these components, includes parametric information. Parametric information may include information regarding curves, equations, and relational data defining the shape of each of the components, i.e., CAD parts 311 to 313. In various embodiments, herein, such parametric information, or variables related to each of the components, can be manipulated or edited, e.g., by processor(s) 104, to alter, update, or otherwise modify or change the shape, appearance, volume, or otherwise dimensions or more of the components (e.g., CAD parts 311 to 313), in order to make the parts fit together or otherwise form a complete or wholly formed virtual product and/or product package. … automatic imaging asset assembly script 302 assembles CAD parts 311 to 313, e.g., by closing (314) the lid/cap (CAD part 311) and orients the label (CAD part 313) to the body (CAD part 312) to form the virtual shampoo bottle (i.e., the parametric-based CAD model 310). The closing and attaching are performed in 3D space based on the parametric information and relation of such data among each of the CAD components (CAD parts 311 to 313). In this way, automatic imaging asset assembly script 302 assembles the appropriate CAD components, corresponding with the predefined design shape, by performing automatic imaging mapping, positioning, or otherwise correlation by closing the cap/lid, fixing orientation of the various CAD components with respect to one another, which includes, assembling the bottle, cap, and label.” [0066-0067] “the polygonal orientation script may check each of CAD parts 311 to 313 for accurate sizing and/or positioning with respect to one another so that each of CAD parts 311 to 313 form a complete or whole (and accurately scaled) version of a real-world product or product package (e.g., the shampoo bottle as described for FIGS. 3A-3C).” [0074] “the visualization editor is configured to load, on a graphical display (e.g., terminal 109), any one or more of the one or more CAD components (e.g., CAD parts 311 to 313), parametric information associated with polygonal model 410, the parametric-based CAD model 410 itself, a polygonal model (e.g., a polygonal model 410), or other visualizable or renderable images or assets, including a UV coordinate mapping, a virtual product label, or a virtual 3D model, as described herein. Each of these imaging assets or models may be manipulated or changed in the visualization editor and applied the polygonal model 410 to create new, different, or updated designs or changes. Such changes may include, by way of non-limiting example, changes or manipulations to scale, size, color, texture, position, orientation, etc.” [0076])
15. With reference to claim 8, Cheshire teaches gloss adjustment data. ( “one or more digital surface finish artifacts of a virtual material library 520, as selected from the 2D imaging assets, may be applied (522) to a polygonal model. For example, as shown in FIG. 5A, the physical materials/CMF 318 are applied (522), by processor(s) 104, to high resolution polygonal model 510H generate a virtual 3D model 810M of the real-world product or product package (e.g., shampoo bottle). Physical materials/CMF 318 may include product surface textures, print finishes, colors, appearances, and finishes (e.g., smooth, shiny, water, wood, metal, grain, etc.). Such Physical materials/CMF 318 may be stored in CAD XML file 350 for access by polygonal model generation script 502, processor(s) 104, and/or a 3D software of a visualization editors. The physical materials/CMF 318 values are applied to high resolution polygonal model 510H by adding the surface textures, print finishes, colors, appearances, etc. to the surface or other area of high resolution polygonal model 510H.” [0086] “color palettes 754, which may include different color sets, such as pre-defined color sets of matching or complementary colors for application to products and/or packages, may be loaded, by processor(s) 104, and applied, or used to paint, high resolution polygonal model 510H. In some embodiments, a user may select colors or color palettes from a visualization editor for application to high resolution polygonal model 510H, including to the materials, finishes, artwork, or other surface changes applied to high resolution polygonal model 510H as described herein.” [0105])
Cheshire does not explicitly teach the stereoscopic image creation means further includes fourth stereoscopic image creation means that creates a fourth stereoscopic image by superimposing on the third stereoscopic image. This is what Hosenpud teaches (“Either or both of the displays 150A and 150B may present (display) stereoscopic images for viewing by the user. By presenting stereoscopic images, the display(s) 150 may present a 3D scene for the user. This 3D scene may be considered or referred to as an illusion or simulated 3D because the actual provided images are 2D, but the scene is conveyed in 3D via the user's interpretation of the provided images via stereoscopic effects.” [0074] “webpages have predominantly revolved around presenting two-dimensional (2D) content, e.g., text, images, videos, and so forth, to end users. However, the introduction of the hypertext markup language (HTML) canvas element and Web Graphics Library (WebGL) has enabled webpages to present three-dimensional (3D) content as well.” [0105] “one or more layout properties of a WebGL canvas may be specified to allow for integration of 3D content with 2D content within a webpage. … the WebGL canvas may be specified with a transparent background (e.g., to allow the WebGL canvas to be overlaid on top of or in front of rendered 2D content and still retain visualization of the rendered 2D content). In some embodiments, the WebGL canvas may be specified as full screen such that a virtual beam associated with the user input device may be rendered to appear that the virtual beam is interacting with the web browser's user interface and scrollbars (anything outside of the browser's client viewport area). FIG. 7 illustrates a block diagram of an example of a method for rendering a webpage that includes both 2D and 3D content, according to some embodiments. The method shown in FIG. 7 may be used in conjunction with any of the systems or devices shown in the above Figures, among other devices. In various embodiments, some of the method elements shown may be performed concurrently, in a different order than shown, or may be omitted. … a webpage including one or more layers may be rendered on a surface of a display device, such as one of the display devices described above. The layers may include a first layer associated with 3D content, e.g., a WebGL canvas, and a second layer associated with 2D content. In some embodiments, a single WebGL canvas may be configured to support multiple instances of 3D content. … the rendering may be a 3D stereoscopic rendering.” [0111-0113]) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Hosenpud into Cheshire, in order to interact with 2D content.
16. With reference to claim 9, Cheshire teaches the two-dimensional image data is data to be printed on a sheet-like material disposed on a surface of the container, the sheet-like material has a first print region having a first side and a second side opposing the first side, and a predetermined warning is issued when at least one of conditions (1) to (3) below is satisfied: (1) a difference in brightness between the two-dimensional image data in a first edge area including the first side and the two-dimensional image data in a second edge area including the second side is a predetermined value or more; (2) a difference in chromaticity between the two-dimensional image data in the first edge area including the first side and the two-dimensional image data in the second edge area including the second side is a predetermined value or more; and (3) a difference in chroma saturation between the two-dimensional image data in the first edge area including the first side and the two-dimensional image data in the second edge area including the second side is a predetermined value or more. (“GTIN identifiers 202 may be used to look up product and/or package information in a database (e.g., database 105 and/or memory 106). Such information may include CAD components (e.g., a CAD part 311, a CAD part 312, and a CAD part 313) or other information (e.g., label position 316, physical and/or color-material-finishes (CMF) data or libraries (e.g., including chromatic/color, tactile and decorative identity of a design of a product/package), referred to herein as physical materials/CMF 318, weights/measurements 319, or other information as described herein, of products and/or packages, or parts thereof) that will be used to construct virtual 3D models, or other models or assets, as described herein. Physical materials/CMF 318 may also include meta data such as formula(s) and/or ingredients corresponding to physical characteristics of, or the making and/or manufacturing of, products and/or packages, or parts thereof as described herein.” [0062] “low resolution polygonal model 510L may be generated quickly in order to determine if any errors have been made or to determine whether the design meets expectations. If not, a new low resolution polygonal model 510L may be regenerated as required or desired to fix errors, make adjustments, or design elements associated with the design of the real-world product and/or package (e.g., a shampoo bottle).” [0082] “one or more digital surface finish artifacts of a virtual material library 520, as selected from the 2D imaging assets, may be applied (522) to a polygonal model. For example, as shown in FIG. 5A, the physical materials/CMF 318 are applied (522), by processor(s) 104, to high resolution polygonal model 510H generate a virtual 3D model 810M of the real-world product or product package (e.g., shampoo bottle). Physical materials/CMF 318 may include product surface textures, print finishes, colors, appearances, and finishes (e.g., smooth, shiny, water, wood, metal, grain, etc.). Such Physical materials/CMF 318 may be stored in CAD XML file 350 for access by polygonal model generation script 502, processor(s) 104, and/or a 3D software of a visualization editors. The physical materials/CMF 318 values are applied to high resolution polygonal model 510H by adding the surface textures, print finishes, colors, appearances, etc. to the surface or other area of high resolution polygonal model 510H.” [0086])
17. With reference to claim 10, Cheshire teaches the two-dimensional image data is data to be printed on a sheet-like material disposed on a surface of the container, the sheet-like material has a first print region having a first side and a second side opposing the first side, and a second print region having a third side adjacent to the second side of the first print region and a fourth side opposing the third side, and a predetermined warning is issued when at least one of conditions (1) to (3) below is satisfied: (1) a difference in brightness between the two-dimensional image data in a second edge area including the second side of the first print region and the two-dimensional image data in a third edge area including the third side of the second print region is a predetermined value or more; (2) a difference in chromaticity between the two-dimensional image data in the second edge area including the second side of the first print region and the two-dimensional image data in the third edge area including the third side of the second print region is a predetermined value or more; and (3) a difference in chroma saturation between the two-dimensional image data in the second edge area including the second side of the first print region and the two-dimensional image data in the third edge area including the third side of the second print region is a predetermined value or more. (“GTIN identifiers 202 may be used to look up product and/or package information in a database (e.g., database 105 and/or memory 106). Such information may include CAD components (e.g., a CAD part 311, a CAD part 312, and a CAD part 313) or other information (e.g., label position 316, physical and/or color-material-finishes (CMF) data or libraries (e.g., including chromatic/color, tactile and decorative identity of a design of a product/package), referred to herein as physical materials/CMF 318, weights/measurements 319, or other information as described herein, of products and/or packages, or parts thereof) that will be used to construct virtual 3D models, or other models or assets, as described herein. Physical materials/CMF 318 may also include meta data such as formula(s) and/or ingredients corresponding to physical characteristics of, or the making and/or manufacturing of, products and/or packages, or parts thereof as described herein.” [0062] “low resolution polygonal model 510L may be generated quickly in order to determine if any errors have been made or to determine whether the design meets expectations. If not, a new low resolution polygonal model 510L may be regenerated as required or desired to fix errors, make adjustments, or design elements associated with the design of the real-world product and/or package (e.g., a shampoo bottle).” [0082] “one or more digital surface finish artifacts of a virtual material library 520, as selected from the 2D imaging assets, may be applied (522) to a polygonal model. For example, as shown in FIG. 5A, the physical materials/CMF 318 are applied (522), by processor(s) 104, to high resolution polygonal model 510H generate a virtual 3D model 810M of the real-world product or product package (e.g., shampoo bottle). Physical materials/CMF 318 may include product surface textures, print finishes, colors, appearances, and finishes (e.g., smooth, shiny, water, wood, metal, grain, etc.). Such Physical materials/CMF 318 may be stored in CAD XML file 350 for access by polygonal model generation script 502, processor(s) 104, and/or a 3D software of a visualization editors. The physical materials/CMF 318 values are applied to high resolution polygonal model 510H by adding the surface textures, print finishes, colors, appearances, etc. to the surface or other area of high resolution polygonal model 510H.” [0086])
18. With reference to claim 11, Cheshire teaches the container data storage means stores a plurality of three-dimensional shape data, and the image display system further comprises container selection means that selects three-dimensional shape data of a container requested by the customer from among the plurality of three-dimensional shape data. (“As described herein a “memory” may refer to either memory 106 and/or database 105. Such memory may be configured to store 2D imaging assets and 3D imaging assets accessible by processor(s) 104, scripts, application, or other software, … database 105 may be a product lifecycle management (PLM) database or system. Generally, a PLM database or system is implemented as an information management system that can integrate data, processes, and other business systems within an enterprise or platform, such as the platform depicted for 3D modeling system 100. A PLM database or system generally includes software for managing information (e.g., 3D imaging assets and 2D imaging assets) throughout an entire lifecycle of a product/package in an efficient and cost-effectivities manner. The lifecycle may include lifecycle stages from ideation, design and manufacture, through service and disposal. In some embodiments, database 105 may store digital PLM objects (e.g., digital 3D imaging assets and/or 2D imaging assets as described herein). Such digital objects or assets can represent a real-world physical parts, assemblies(s), or documents, customer requirements or supplier parts, a change process, and/or other data types relating to a lifecycle management and development of a product and/or package. For example, digital objects or assets can include computer-aided design (CAD) file(s) that depict or describe (e.g., via measurements, sizes, etc.) parts, components, or complete (or partially complete) models or designs of products and/or packages. Generally, non-CAD files can also be included database 105. Such non-CAD files can include text or data files describing or defining parts, components, and/or product or package specifications, vendor datasheets, or emails relating to a design.” [0042-0043] “processor(s) 104 may store virtual 3D model(s) in memory 106 and/or database 105 such that virtual 3D model(s) are accessible to an automatic imaging asset assembly script or a visualization editor.” [0045] “a predefined design shape corresponding to a real-world product or product package (e.g., a shampoo bottle with a label or package of toilet paper with a wrapper) is selected for submission and search of PLM database system 210. As shown in the embodiment of FIG. 2, a user may select the predefined design shape from a user interface 209. In other embodiments, one or more predefined design shape(s) may be loaded into a script for submission to the PLM database system 210 without user selection. Various types of shapes may be selected or used as the predefined design shape 3D.” [0058] “one or more digital surface finish artifacts of a virtual material library 520, as selected from the 2D imaging assets, may be applied (522) to a polygonal model. For example, as shown in FIG. 5A, the physical materials/CMF 318 are applied (522), by processor(s) 104, to high resolution polygonal model 510H generate a virtual 3D model 810M of the real-world product or product package (e.g., shampoo bottle). Physical materials/CMF 318 may include product surface textures, print finishes, colors, appearances, and finishes (e.g., smooth, shiny, water, wood, metal, grain, etc.). Such Physical materials/CMF 318 may be stored in CAD XML file 350 for access by polygonal model generation script 502, processor(s) 104, and/or a 3D software of a visualization editors. The physical materials/CMF 318 values are applied to high resolution polygonal model 510H by adding the surface textures, print finishes, colors, appearances, etc. to the surface or other area of high resolution polygonal model 510H.” [0086])
19. With reference to claim 12, Cheshire teaches the customer is capable of settling purchase of a container, based on the first two-dimensional image data and the second two-dimensional image data having been determined. (“Such virtual packages and products can be used in the development process of a real-world product and packaging for a variety of purposes, including consumer testing, product modeling, management decision making, customer decision making, quality control, etc., all of which can reduce the development lifecycle dramatically.” [0005] “database 105 may be a product lifecycle management (PLM) database or system. Generally, a PLM database or system is implemented as an information management system that can integrate data, processes, and other business systems within an enterprise or platform, such as the platform depicted for 3D modeling system 100. A PLM database or system generally includes software for managing information (e.g., 3D imaging assets and 2D imaging assets) throughout an entire lifecycle of a product/package in an efficient and cost-effectivities manner. The lifecycle may include lifecycle stages from ideation, design and manufacture, through service and disposal. In some embodiments, database 105 may store digital PLM objects (e.g., digital 3D imaging assets and/or 2D imaging assets as described herein). Such digital objects or assets can represent a real-world physical parts, assemblies(s), or documents, customer requirements or supplier parts, a change process, and/or other data types relating to a lifecycle management and development of a product and/or package. For example, digital objects or assets can include computer-aided design (CAD) file(s) that depict or describe (e.g., via measurements, sizes, etc.) parts, components, or complete (or partially complete) models or designs of products and/or packages. Generally, non-CAD files can also be included database 105. Such non-CAD files can include text or data files describing or defining parts, components, and/or product or package specifications, vendor datasheets, or emails relating to a design.” [0043] “artwork 704 may be split into masks or polygons in order for processor(s) 104 to apply different finishes or colors swatches to match materials. If masks are used, processor(s) 104 may select corresponding virtual materials based artwork 704 that is applied to high resolution polygonal model 510H.” [0100] “high resolution polygonal model 510H and its one or more digital surface finish artifacts, as generated with physical materials/CMF 318, virtual material library 520 back plates or background images 714, and/or lighting effects 716.” [0103])
Conclusion
20. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michelle Chin whose telephone number is (571)270-3697. The examiner can normally be reached on Monday-Friday 8:00 AM-4:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http:/Awww.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Kent Chang can be reached on (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is (571)273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https:/Awww.uspto.gov/patents/apply/patent- center for more information about Patent Center and https:/Awww.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHELLE CHIN/
Primary Examiner, Art Unit 2614
Prosecution Insights