Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 08/06/2026 has been entered.
Response to Amendment
Applicant’s amendments and remarks submitted 08/06/2026 have been entered and considered, Claims 1, 18, 22, 25 are amended.
Response to Arguments
Applicant’s arguments filed on 08/06/2026 have been fully considered but are moot because they don’t apply to the reference(s)/combination(s) in the current rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 7-12, 14-15, 17, 22, 25-26 are rejected under 35 U.S.C. 103 as being unpatentable over Haveman et al (US20170200302) in view of Scranton et al (US9367950) further in view of Maher et al. ("Designworld: an augmented 3D virtual world for multidisciplinary, collaborative design." Proceedings of CAADRIA 2006 (2006): 133-142).
Regarding Claim 1. Haveman teaches A Computer Generated Imagery (CGI) system (Haveman, abstract, the invention describes a method and system for high-performance real-time adjustment of colors and patterns on one or more pre-selected elements in a playing video, interactive 360° content, or other image, using graphics processing unit ("GPU") shader code to process the asset per-pixel and blend in a target color or pattern based on prepared masks and special metadata lookup tables encoded visually in the video file. The method and system can generate asset-specific optimized GPU code that is generated from templates. Pixels are blended into the target asset using the source image for shadows and highlights, one or more masks, and various metadata lookup-tables, such as a texture lookup-table that allows for changing or adding patterns, z-depth to displace parts of the image, or normals to calculate light reflection and refraction.) comprising:
a processor having software executing thereon, said software providing a user interface which allows a user to select two or more asset files which are pre-configured CGI asset files to combine the two or more asset files into a CGI asset (Haveman, [0017] With reference to FIG. 1, the hardware associated with various aspects of the present invention will be described. Assets, such as videos, 360° content or images for use with the present invention, are generated at an asset generation computer 1, as will be described in more detail in connection with FIGS. 2A-2C through 5A-5B. The assets are delivered to a user computer 2, typically through a standard Internet connection…. The user computer 2 contains a standard CPU 3, and a graphics processing unit, or GPU 4.
[0082] Another application of the present invention to the display of 360° content will be described with reference to FIGS. 4A-4B, 7 and 11A-11E. Here, an asset, namely a bottle, is prepared as described with reference to FIGS. 4A-4E. Playback is commenced, at block 34, FIG. 7, and the bottle is displayed within a user interface, an example of which is shown in FIG. 11A, The user interface allows the user to select different patterns or textures (in this case, labels) at input 77. The user interface also allows the user to select different backgrounds and settings at input 78, and different orientations (rotations through the two axes discussed with reference to FIGS. 4C-4E, as well as a rotation through a third axis, normal to the plane of the screen) and positions (translations), at input 79.
[0083] For example, the user may select a particular label 80A-80C, at input 77, which will be applied to the region and in the manner specified by regions 20E-20I,);
a first asset is a background including image content and is one of the two or more asset files, and a second asset is an object and is one of the two or more asset files (Haveman, [0082] Another application of the present invention to the display of 360° content will be described with reference to FIGS. 4A-4B, 7 and 11A-11E. Here, an asset, namely a bottle, is prepared as described with reference to FIGS. 4A-4E. Playback is commenced, at block 34, FIG. 7, and the bottle is displayed within a user interface, an example of which is shown in FIG. 11A, The user interface allows the user to select different patterns or textures (in this case, labels) at input 77. The user interface also allows the user to select different backgrounds and settings at input 78, and different orientations (rotations through the two axes discussed with reference to FIGS. 4C-4E, as well as a rotation through a third axis, normal to the plane of the screen) and positions (translations), at input 79. Unlike the full-frame video described with reference to FIGS. 10A-10C, the display of FIGS. 11A-11E can be static until the user modifies a parameter through the user interface. The player will check to see if such has occurred, in block 35, FIG. 7, and if so, processing proceeds through blocks 37-40, FIG. 7, to update the frame of the display.
As shown in Figs 11D-11E, the background is an image of beach.);
the user interface allows the user to arrange the first and second assets and to select perspective information which perspective information is indicative of: ground plane, surface information, point of view position, lighting, configuration or combinations thereof associated with the background (Haveman, [0082] Another application of the present invention to the display of 360° content will be described with reference to FIGS. 4A-4B, 7 and 11A-11E. Here, an asset, namely a bottle, is prepared as described with reference to FIGS. 4A-4E. Playback is commenced, at block 34, FIG. 7, and the bottle is displayed within a user interface, an example of which is shown in FIG. 11A, The user interface allows the user to select different patterns or textures (in this case, labels) at input 77. The user interface also allows the user to select different backgrounds and settings at input 78, and different orientations (rotations through the two axes discussed with reference to FIGS. 4C-4E, as well as a rotation through a third axis, normal to the plane of the screen) and positions (translations), at input 79. Unlike the full-frame video described with reference to FIGS. 10A-10C, the display of FIGS. 11A-11E can be static until the user modifies a parameter through the user interface. The player will check to see if such has occurred, in block 35, FIG. 7, and if so, processing proceeds through blocks 37-40, FIG. 7, to update the frame of the display.
Although Haveman didn’t explicitly teach perspective information, Haveman did describe the rotation parameter with reference to a plane. It is obvious to a person with ordinary skill in the art that perspective information can be calculated from the rotation parameters. Therefore, rotation parameters implies perspective information.);
wherein the software uses the perspective information to modify how
image content of the second asset is displayed on and relative to the first asset on sthe user interface and the user interface allows the second asset to be user
manipulated consistent with the perspective information to create the CGI
asset (Haveman, [0084] The user can also select different backgrounds and settings for the bottle, essentially to place it anywhere he or she desires. For example, in FIG. 11D, the user has selected a background for the bottle on a beach, and its appearance will be a function of the reflection and refraction characteristics defined by the neutral, transparency, diffusion, intensity and normal regions 20A, 20C, 20D, 20E and 201, in
combination with the background, as described above. This can be compared to FIG. 11E, where the user has translated the bottle, at input 79, so that it obscures the jogger in the selected background. Different refraction effects can be seen at 82, for example, where the jogger and his shadow can be seen as refracted through the semitransparent bottle. Again, this effect is highly realistic and is produced automatically for any background setting selected by the user.
Therefore, in Fig 11D & 11E, the object bottle is put in different perspective relative to background image.).
Haveman fails to explicitly teach, however, Scranton teaches create the CGI asset and state data such that said software automatically creates said state data (Scranton, abstract, the invention describes method for providing a user virtual reality experience to visualize a three-dimensional design produced using three-dimensional design software are disclosed. The user may be enabled to visualize the three-dimensional design through a virtual reality application while the design is in progress. Changes to the three-dimensional design may be obtained dynamically and virtual reality information may be determined based on the dynamically obtained changes. The determined virtual reality information may be provided to the user for implementation of a virtual reality visualizing the three-dimensional design on a client device associated with the user.
Col 6, line 1-25, The virtual reality application 202 may be configured to implement an instance of a virtual space based on the virtual reality information provided by the server 102. The instance of the virtual space may represent a three-dimensional visualization of the 3D design produced by the 3D design software 204. The instance of the virtual space may reflect a state of the 3D design as it is being completed towards finalization. Implementation of the instance of the virtual space by the virtual reality application 202 may involve determining a view of the virtual space, and/or other information that describes the virtual space. Expressions of the instance of virtual space on the client device 104 may facilitate presentation of views of the virtual space to the user associated with the client device 104. In some implementations, expressions of the instance executed on the client device 104 may be configured to simply present views of the virtual space based on the virtual reality information (e.g., via streaming information, object/position information, and/or other virtual reality information) received from server 102. The view presented on the client device 104 may correspond to a location in the virtual space (e.g., the location from which the view is taken, the location the view depicts, and/or other locations), a zoom ratio, a dimensionality of objects, a point-of-view, and/or view parameters. One or more of the view parameters may be selectable by the user.
Col 6, line 54-67, Within the instance of the virtual space implemented by virtual reality application 202, users may traverse the virtual space, manipulate the virtual objects in the virtual space, perform operation, exercise control, and/or engage any other types of interactions with the virtual space. The users may participate in the instance of the virtual space by controlling one or more of the available user controlled elements in the virtual space.
Col 8, line 1-20, FIG. 4 illustrates one example of facilitating the communication shown in FIG. 3 by the client device 104. It will be described with reference to FIG. 3. As shown in this example, the client device 104 shown in FIG. 3 may include a file storage unit 402 and a memory unit 404. In this example, the 3D design software 204 is configured to retrieve and save changes 406 of the 3D design made by the user to the file storage unit 402 via the memory unit 404. The changes 406 may include changes (including addition and deletion) made to the two dimensional and/or three-dimensional models of the 3D design, attribute changes in BIM specification, changes in drawings, and/or any other types of changes. In this example, as illustrated, the listener/service component 302 may be configured to detect the changes 406 as they are saved to the file storage unit 402 by the 3D design software 204. This may involve tracking the state change of the files associated with the 3D design software 204 storing in the file storage unit 402, such as changes in date and time of modification, file size, content of the file, and/or any other state changes of those files.).
Haveman and Scranton are analogous art, because they both teach method of generating virtual image by selecting/defining background image and foreground object. Scranton further teaches saving changes to the 3D virtual space/world when user manipulating the virtual objects within the space. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the virtual image generation method (taught by Haveman), to further allow server to save the changed modification to a file on the server (taught in Scranton), so as to conveniently import 3D models from file and display them on 3D visualization interface (Scranton, col 1, line 46-61).
The combination of Haveman and Scranton fails to explicitly teach, however, Maher teaches wherein the CGI asset is downloadable as an image or video file (Maher, abstract, page 133, par 1, page 134, par 1, the paper presents a 3D virtual world environment (DesignWorld) augmented with software agents that provides real-time multi-user collaboration for designers in different locations.
Page 136, par 1, the external model used in the collaborative design environment DesignWorld is compatible with Industry Foundation Classes (IFCs) (IAI 2000). This allows the models to be uploaded from IFC compatible applications such as ArchiCAD for use in collaborative sessions. The agents in DesignWorld extend the platform of a virtual world by maintaining a model of designed objects in an SQL database.
Page 137, par 1, DesignWorld uses the Second Life (www.secondlife.com) virtual environment as the platform for design and collaboration. Second Life allows collaborative manipulation and visualization of shared objects, both synchronously and asynchronously. Designers can select from a range of primitive objects with which to build, which can then be further modified using a number of built-in tools to achieve more complex objects.
Page 138, par 3, The object-property viewer uses discipline and relationship information stored in the external model to display non-geometric properties of objects that are not visible in the 3D virtual world. Designers view non-geometric properties by clicking on the desired object in Second Life then clicking the view button in the web page to send a request to the object-property viewer agent. The agent retrieves the properties from the external model and displays them in the web browser. The non-geometric properties that can be viewed are specific to the discipline to which the object belongs. These properties are attached to the object by agents in DesignWorld. At present, non-geometric properties are not imported from the IFC model but could be in the future.
Page 140, par 1, Designers upload their models by using the web-browser to send a request to the builder agent to build the objects in a specified .ifc file. This file is then uploaded to an EDM database and converted into SQL format and loaded into the external model. The builder agent then reads data about individual objects from the external model and constructs the objects in the 3D world. Figure 5 shows a simple house model imported from a .ifc file.
Therefore, users modify the 3D virtual object. The corresponding changes (properties attached to the object) can be attached to the IFC model file.).
Haveman, Scranton and Maher are analogous art, because they all teach method of generating virtual image based on user’s selecting/configuring/editing virtual image object. Maher further teaches method of modifying 3D virtual object and save such changes to a file. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the virtual image generation method (taught by Haveman and Scranton), to further use the saving and loading method for the modified virtual object data (taught in Maher), so as to provide a GUI for user to reload the different design later for further modification.
The combination of Haveman, Scranton and Maher further teaches and the state data defines how the first and second assets, as displayed based on the perspective information, are arranged in the CGI asset (Scranton, Col 6, line 1-25, The virtual reality application 202 may be configured to implement an instance of a virtual space based on the virtual reality information provided by the server 102. The instance of the virtual space may represent a three-dimensional visualization of the 3D design produced by the 3D design software 204. The instance of the virtual space may reflect a state of the 3D design as it is being completed towards finalization. Implementation of the instance of the virtual space by the virtual reality application 202 may involve determining a view of the virtual space, and/or other information that describes the virtual space. Expressions of the instance of virtual space on the client device 104 may facilitate presentation of views of the virtual space to the user associated with the client device 104. In some implementations, expressions of the instance executed on the client device 104 may be configured to simply present views of the virtual space based on the virtual reality information (e.g., via streaming information, object/position information, and/or other virtual reality information) received from server 102. The view presented on the client device 104 may correspond to a location in the virtual space (e.g., the location from which the view is taken, the location the view depicts, and/or other locations), a zoom ratio, a dimensionality of objects, a point-of-view, and/or view parameters. One or more of the view parameters may be selectable by the user.) and
wherein the user interface, when provided with the state data, utilizes the state data to obtain and configure the first and second assets at a starting point defined by the state data with the image content of the second asset modified to be displayed in the interface on and relative to the first asset consistent with the state data to allow the background and object shown in the CGI asset to be manipulated in the user interface from the starting point (Maher, page 138, par 1, 4.1. MULTIPLE-VIEWS: The models created by members of different design disciplines are influenced by the different functional concerns of those disciplines. For example, an architect may be concerned with the design of functional spaces within a building while a structural engineer may be concerned with the position of load bearing walls. A single model approach to multidisciplinary design is insufficient for representing these different functional concerns (Rosenman and Gero, 1996). The agents in DesignWorld keep track of the objects created by each discipline in order to maintain information relevant to the different functional concerns of designers from different disciplines. A selection of viewing tools enables designers to view the components relevant to them.
Page 138, par 2, 4.1.1. The Discipline Viewer: The discipline viewer uses object ownership and discipline information stored in the external model to construct different views of a design with respect to the discipline of its designers. Designers can view the parts of a model that are relevant to their discipline by clicking on the appropriate view button in the web-browser to send a request to the discipline viewer agent. The agent retrieves a list of relevant objects from the external model then modifies the transparency of objects in the 3D world so only the relevant objects are visible. Two views of a tower model, an architect’s view and an engineer’s view are shown in Figure 4.
Page 138, par 3, 4.1.2. The Object Property Viewer: The object-property viewer uses discipline and relationship information stored in the external model to display non-geometric properties of objects that are not visible in the 3D virtual world. Designers view non-geometric properties by clicking on the desired object in Second Life then clicking the view button in the web page to send a request to the object-property viewer agent. The agent retrieves the properties from the external model and displays them in the web browser. The non-geometric properties that can be viewed are specific to the discipline to which the object belongs. These properties are attached to the object by agents in DesignWorld. At present, non-geometric properties are not imported from the IFC model but could be in the future.).
Regarding Claim 2. The combination of Haveman, Scranton and Maher further teaches The system of claim 1 wherein: the user interface allows the second asset to be user manipulated by moving a position of the second asset relative to the background such that a change in position of the second asset modifies an orientation of the second asset, lighting of the asset, shadow of the asset or combinations thereof (Haveman, [0084] The user can also select different backgrounds and settings for the bottle, essentially to place it anywhere he or she desires. For example, in FIG. 11D, the user has selected a background for the bottle on a beach, and its appearance will be a function of the reflection and refraction characteristics defined by the neutral, transparency, diffusion, intensity and normal regions 20A, 20C, 20D, 20E and 201, in combination with the background, as described above. This can be compared to FIG. 11E, where the user has translated the bottle, at input 79, so that it obscures the jogger in the selected background. Different refraction effects can be seen at 82, for example, where the jogger and his shadow can be seen as refracted through the semitransparent bottle. Again, this effect is highly realistic and is produced automatically for any background setting selected by the user.
Therefore, in Fig 11D & 11E, the object bottle is put in different perspective relative to background image, when the bottle is moved to a position that obscures the jogger in the background.).
Regarding Claim 7. The combination of Haveman, Scranton and Maher further teaches The system of claim 1 wherein the second asset can be user manipulated to change a color thereof (Haveman, [0079] In conclusion, the system and method described above allow a user, by simply setting a new property (e.g., the color for a region) in a user interface, to change, in real-time, with no latency, the color or other aspect of the configurable element, without requiring multiple, separate videos for each color/pattern combination, or the reloading of an entire new video from a server. This is demonstrated with reference to the examples shown in FIGS. 10A-10C and 11A-11C.).
Regarding Claim 8. The combination of Haveman, Scranton and Maher further teaches The system of claim 1 wherein the second asset can be user manipulated to change a configuration thereof (Haveman, [0080] With reference to FIGS. 3A-3B, 7 and 10A-10C, playback of a video allowing real-time adjustment of color will be described. In this case, playback is commenced, at block 34, FIG. 7, of a video of an asset, namely an athletic shoe, which has been prepared as described in connection with FIGS. 3A-3B. The player displays a user interface 75, FIGS. 10A-10C, in a well-known manner, which allows the user to change the color of the modifiable regions of the shoe, as desired. On a frame by frame basis, video playback is implemented at blocks 35-36, FIG. 7. At block 35, the player checks to see whether the modifiable region has changed based upon the user input of the desired color for the region, based upon the user-controlled cursor, 76, FIGS. 10A-10B. If this is the case, processing proceeds through blocks 37-40, FIG. 7, to update the frame of the playing video, and a new frame is displayed with the selected color, as shown in the sequence of displays in FIGS. 10A-10C.
[0081] Thus, by simply moving the cursor 76 of the user interface 75, the color of the configurable region can be changed in real-time, with no latency, in a highly realistic manner, and without requiring multiple, separate videos for each color/pattern combination, or the reloading of an entire new video from a server. Such facility is extremely useful in many applications, for example, in allowing a potential purchaser to view an item in many different colors or styles, or allowing a product designer to develop new products.).
Regarding Claim 10. The combination of Haveman, Scranton and Maher further teaches The system of claim 1 wherein the user interface is displayed on a display which is remote to the processor via a network (Haveman, [0017] With reference to FIG. 1, the hardware associated with various aspects of the present invention will be described. Assets, such as videos, 360° content or images for use with the present invention, are generated at an asset generation computer 1, as will be described in more detail in connection with FIGS. 2A-2C through 5A-5B. The assets are delivered to a user computer 2, typically through a standard Internet connection.
It is obvious to a person with ordinary skill in the art that a user computer includes a display and connects to a server which includes processor(s), through internet connection.).
Regarding Claim 11. The combination of Haveman, Scranton and Maher further teaches The system of claim 1 wherein the CGI asset contains the state data and the image or video data in one file (Maher, page 138, par 3, The object-property viewer uses discipline and relationship information stored in the external model to display non-geometric properties of objects that are not visible in the 3D virtual world. Designers view non-geometric properties by clicking on the desired object in Second Life then clicking the view button in the web page to send a request to the object-property viewer agent. The agent retrieves the properties from the external model and displays them in the web browser. The non-geometric properties that can be viewed are specific to the discipline to which the object belongs. These properties are attached to the object by agents in DesignWorld. At present, non-geometric properties are not imported from the IFC model but could be in the future.
Therefore, users modify the 3D virtual object. The corresponding changes (properties attached to the object) can be attached to the IFC model file.).
The reasoning for combination of Haveman, Scranton and Maher is the same as described in Claim 1.
Regarding Claim 12. The combination of Haveman, Scranton and Maher further teaches The system of claim 1 wherein the state data is indicative of at least: a position relative to the background, a configuration of the second asset or combinations thereof (Maher, page 138, par 2, 4.1.1. The Discipline Viewer: The discipline viewer uses object ownership and discipline information stored in the external model to construct different views of a design with respect to the discipline of its designers. Designers can view the parts of a model that are relevant to their discipline by clicking on the appropriate view button in the web-browser to send a request to the discipline viewer agent. The agent retrieves a list of relevant objects from the external model then modifies the transparency of objects in the 3D world so only the relevant objects are visible. Two views of a tower model, an architect’s view and an engineer’s view are shown in Figure 4.
Page 138, par 3, The object-property viewer uses discipline and relationship information stored in the external model to display non-geometric properties of objects that are not visible in the 3D virtual world. Designers view non-geometric properties by clicking on the desired object in Second Life then clicking the view button in the web page to send a request to the object-property viewer agent. The agent retrieves the properties from the external model and displays them in the web browser. The non-geometric properties that can be viewed are specific to the discipline to which the object belongs. These properties are attached to the object by agents in DesignWorld. At present, non-geometric properties are not imported from the IFC model but could be in the future.).
The reasoning for combination of Haveman, Scranton and Maher is the same as described in Claim 1.
Regarding Claim 15. The combination of Haveman, Scranton and Maher further teaches The system of claim 1 wherein the user interface is a web based user interface and the processor includes a server (Haveman, [0017] With reference to FIG. 1, the hardware associated with various aspects of the present invention will be described. Assets, such as videos, 360° content or images for use with the present invention, are generated at an asset generation computer 1, as will be described in more detail in connection with FIGS. 2A-2C through 5A-5B. The assets are delivered to a user computer 2, typically through a standard Internet connection…. The user computer 2 contains a standard CPU 3, and a graphics processing unit, or GPU 4.
[0079] In conclusion, the system and method described above allow a user, by simply setting a new property (e.g., the color for a region) in a user interface, to change, in real-time, with no latency, the color or other aspect of the configurable element, without requiring multiple, separate videos for each color/pattern combination, or the reloading of an entire new video from a server. This is demonstrated with reference to the examples shown in FIGS. 10A-10C and 11A-11C.).
Regarding Claim 17. The combination of Haveman, Scranton and Maher further teaches The system of claim 1 wherein the software receives commands via the user interface and converts those commands into commands for the CGI software such that the CGI software generates the CGI asset for display on the user interface based on the commands (Haveman, [0082] Another application of the present invention to the display of 360° content will be described with reference to FIGS. 4A-4B, 7 and 11A-11E. Here, an asset, namely a bottle, is prepared as described with reference to FIGS. 4A-4E. Playback is commenced, at block 34, FIG. 7, and the bottle is displayed within a user interface, an example of which is shown in FIG. 11A, The user interface allows the user to select different patterns or textures (in this case, labels) at input 77. The user interface also allows the user to select different backgrounds and settings at input 78, and different orientations (rotations through the two axes discussed with reference to FIGS. 4C-4E, as well as a rotation through a third axis, normal to the plane of the screen) and positions (translations), at input 79.
[0083] For example, the user may select a particular label 80A-80C, at input 77, which will be applied to the region and in the manner specified by regions 20E-20I.
Therefore, user’s selection commands are used to generate the virtual environment shown in FIGS. 4A-4B and 11A-11E.).
Claim 22 is similar in scope as Claim 1, and thus is rejected under same rationale. Claim 22 further requires:
wherein the file of the CGI asset is configured to be opened from the user interface at any computing device (Maher, page 135, par 1, DesignWorld uses a client-server architecture, shown in Figure 2, to provide design and collaboration tools. Designers interact with DesignWorld using a client browser. The client browser, also depicted in Figure 1, has two components, a 3D world window and a web window. The 3D world window is the primary interface through which designers can build representations of design artefacts. We use the Second Life client browser for this purpose. Second Life is an online persistent space, created and evolved by its users with built-in content creation tools.) that can display and manipulate the user interface to allow further modification of the CGI asset with a starting point of the CGI asset determined by the state information and the CGI asset is configured to be opened in the fixed state outside the user interface to display the CGI asset as downloaded from the user interface in the fixed state (Maher, page 136, par 1, The DesignWorld external model is a MySQL database. The database schema is compatible with Industry Foundation Classes (IFCs) (IAI, 2000) so a model can be uploaded from IFC compatible applications such as ArchiCad for use in collaborative sessions. The external model contains project information for a group of objects, and for each object there is discipline, versioning and relationship information.
Page 138, par 3, 4.1.2. The Object Property Viewer: The object-property viewer uses discipline and relationship information stored in the external model to display non-geometric properties of objects that are not visible in the 3D virtual world. Designers view non-geometric properties by clicking on the desired object in Second Life then clicking the view button in the web page to send a request to the object-property viewer agent. The agent retrieves the properties from the external model and displays them in the web browser. The non-geometric properties that can be viewed are specific to the discipline to which the object belongs. These properties are attached to the object by agents in DesignWorld. At present, non-geometric properties are not imported from the IFC model but could be in the future.).
The reasoning for combination of Haveman, Scranton and Maher is the same as described in Claim 1.
Claim 25 is similar in scope as Claim 1, and thus is rejected under same rationale.
Claim 26 is similar in scope as Claim 2, and thus is rejected under same rationale.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Haveman et al (US20170200302) in view of Scranton et al, Maher et al further in view of Garbut et al (US10987587).
Regarding Claim 14. The combination of Haveman, Scranton and Maher fails to explicitly teach, however, Garbut teaches The system of claim 1 wherein the state data is metadata (Garbut, abstract, the invention describes a system and method for building and rendering in-game objects. In some embodiments, the system includes a plurality of metadata records describing available assets, wherein each metadata record comprises an asset identifier, which identifies an asset, and a property tag. In some embodiments, the system receives the metadata asset record or the asset identified by the metadata asset record from an asset interface to instruct a rendering engine to load the asset identified by the metadata asset record into a 3D environment.
Col 6, line 56-67, Property Tags: In one embodiment, the metadata further includes Property Tags, which as previously discussed, permit developers to include open-ended and unrestricted characteristics about the underlying asset. These tags can, for example, name the type of the object, e.g., hat, shoe, gun, etc., or they can provide a quality characteristic, e.g., new, old, preppy, grungy, red, green. These Property Tags are used for a wide range of purposes. Advantageously, due to their open-ended nature they can extend functionality without requiring changes to the code.
Col 10, line 50-67, Expression Data: Expression data is an additional optional metadata field that can be specified for models. It is used to convey artist defined control information to the animation system. An example of expression data for a hat model may include driving the animation of a character's hair to squash it down and not intersect the geometry once the hat has been placed on the character's head.).
Haveman, Scranton, Maher and Garbut are analogous art, because they all teach method of generating virtual image by selecting/editing background image and foreground object. Garbut further teaches using metadata to store related information of corresponding virtual object. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the virtual image generation method (taught by Haveman, Scranton and Maher), to further use metadata to store data such as state information (taught in Garbut), so as to allow virtual object to be created with different configurations stored in metadata (Garbut, col 1, line 30-49).
Claims 3-6, 18, 27-29 are rejected under 35 U.S.C. 103 as being unpatentable over Haveman et al (US20170200302) in view of Scranton et al, Maher et al further in view of Wang et al (US20190082118).
Regarding Claim 3. The combination of Haveman, Scranton and Maher fails to explicitly teach, however, Wang teaches The system of claim 1 wherein the first asset is a three-dimensional background (Wang, abstract, the invention describes methods for generating AR self-portraits or "AR selfies." In an embodiment, a method comprises: capturing, by a first camera of a mobile device, live image data, the live image data including an image of a subject in a physical, real-world environment; receiving, by a depth sensor of the mobile device, depth data indicating a distance of the subject from the camera in the physical, real-world environment; receiving, by one or more motion sensors of the mobile device, motion data indicating at least an orientation of the first camera in the physical, real-world environment; generating a virtual camera transform based on the motion data, the camera transform for determining an orientation of a virtual camera in a virtual environment; and generating a composite image data, using the image data, a matte and virtual background content selected based on the virtual camera orientation.
[0042] FIGS. 3C and 3D illustrate graphical user interfaces with different background scenes selected and showing a recording view and full-screen playback view, according to an embodiment. In FIG. 3C, a recording view is shown where user 302c has selected a virtual background 303c.
[0069] Process 900 continues by receiving a virtual background content (903) from storage. For example, the virtual background content can be a 2D image, 3D image or 360° video. The virtual background content can be selected by the user through a GUI. The virtual background content can be extracted or sampled from any desired virtual environment, such as a famous city or cartoon environment with animated cartoon characters and objects.
Haveman, [0016] FIGS. 11A-11E illustrate a display of 360° content and examples of adjustments to texture, reflection and refraction based on user-selected patterns, orientations and backgrounds, in accordance with the present invention.).
Haveman, Scranton, Maher and Wang are analogous art, because they all teach method of generating virtual image by selecting/editing background image and foreground object. Wang further teaches selecting 3D background image. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the virtual image generation method (taught by Haveman, Scranton and Maher), to further use 3D background image (taught in Wang), so as to create a live view of physical, real-world environment whose elements are “augmented” by computer generated sensory input such as sound, video or graphics (Wang, [0003]).
Regarding Claim 4. The combination of Haveman, Scranton, Maher and Wang further teaches The system of claim 1 wherein the user interface allows the perspective information to be user modifiable to modify how the background is displayed (Haveman, [0082] Another application of the present invention to the display of 360° content will be described with reference to FIGS. 4A-4B, 7 and 11A-11E. Here, an asset, namely a bottle, is prepared as described with reference to FIGS. 4A-4E. Playback is commenced, at block 34, FIG. 7, and the bottle is displayed within a user interface, an example of which is shown in FIG. 11A, The user interface allows the user to select different patterns or textures (in this case, labels) at input 77. The user interface also allows the user to select different backgrounds and settings at input 78, and different orientations (rotations through the two axes discussed with reference to FIGS. 4C-4E, as well as a rotation through a third axis, normal to the plane of the screen) and positions (translations), at input 79.
Wang, [0037] FIGS. 3A and 3B are graphical user interfaces for recording AR selfies, according to an embodiment. Referring to FIG. 3A, AR selfie GUI 300 includes viewport 301 displaying a composite video frame that includes selfie subject 302a and virtual background content 303a. A "cartoon" special effect has been applied to the composite video to create an interesting effect and to hide artifacts from the alpha compositing process. Although a single composite video frame is shown, it should be understood that viewport 301 is displaying a live video feed (e.g., 30 frames/second), and if the orientation of the real-world camera view direction changes, virtual background 303a will also seamlessly change to show a different portion of the virtual environment. This allows the user to "look around" the visual environment by changing the view direction of the realworld camera.).
The reasoning for combination of Haveman, Scranton, Maher and Wang is the same as described in Claim 3.
Regarding Claim 5. The combination of Haveman, Scranton, Maher and Wang further teaches The system of claim 4 wherein modification of the perspective information modifies how the second asset is displayed relative to the first asset (Haveman, [0082] Another application of the present invention to the display of 360° content will be described with reference to FIGS. 4A-4B, 7 and 11A-11E. Here, an asset, namely a bottle, is prepared as described with reference to FIGS. 4A-4E. Playback is commenced, at block 34, FIG. 7, and the bottle is displayed within a user interface, an example of which is shown in FIG. 11A, The user interface allows the user to select different patterns or textures (in this case, labels) at input 77. The user interface also allows the user to select different backgrounds and settings at input 78, and different orientations (rotations through the two axes discussed with reference to FIGS. 4C-4E, as well as a rotation through a third axis, normal to the plane of the screen) and positions (translations), at input 79.
[0084] The user can also select different backgrounds and settings for the bottle, essentially to place it anywhere he or she desires. For example, in FIG. 11D, the user has selected a background for the bottle on a beach, and its appearance will be a function of the reflection and refraction characteristics defined by the neutral, transparency, diffusion, intensity and normal regions 20A, 20C, 20D, 20E and 201, in combination with the background, as described above. This can be compared to FIG. 11E, where the user has translated the bottle, at input 79, so that it obscures the jogger in the selected background. Different refraction effects can be seen at 82, for example, where the jogger and his shadow can be seen as refracted through the semitransparent bottle. Again, this effect is highly realistic and is produced automatically for any background setting selected by the user.
Therefore, in Fig 11D & 11E, the object bottle is put in different perspective relative to background image, when the bottle is moved to a position that obscures the jogger in the background.).
Regarding Claim 6. The combination of Haveman, Scranton, Maher and Wang further teaches The system of claim 3 wherein the first asset is a three dimensional background and the three dimensional background and perspective information is used to modify how the second asset is displayed in order to provide a reflection of at least part of the background on the second asset (Wang, [0069] Process 900 continues by receiving a virtual background content (903) from storage. For example, the virtual background content can be a 2D image, 3D image or 360° video. The virtual background content can be selected by the user through a GUI. The virtual background content can be extracted or sampled from any desired virtual environment, such as a famous city or cartoon environment with animated cartoon characters and objects.
Haveman, [0084] The user can also select different backgrounds and settings for the bottle, essentially to place it anywhere he or she desires. For example, in FIG. 11D, the user has selected a background for the bottle on a beach, and its appearance will be a function of the reflection and refraction characteristics defined by the neutral, transparency, diffusion, intensity and normal regions 20A, 20C, 20D, 20E and 201, in combination with the background, as described above. This can be compared to FIG. 11E, where the user has translated the bottle, at input 79, so that it obscures the jogger in the selected background. Different refraction effects can be seen at 82, for example, where the jogger and his shadow can be seen as refracted through the semitransparent bottle. Again, this effect is highly realistic and is produced automatically for any background setting selected by the user.
Therefore, in Fig 11D & 11E, the object bottle is put in different perspective relative to background image, when the bottle is moved to a position that obscures the jogger (part of the background) in the background. The shadow of the jogger can be seen through the bottle in Fig 11E.).
The reasoning for combination of Haveman, Scranton, Maher and Wang is the same as described in Claim 3.
Claim 18 is similar in scope as Claim 1 & 3-5, and thus is rejected under same rationale.
Claim 27 is similar in scope as Claim 3, and thus is rejected under same rationale.
Claim 28 is similar in scope as Claim 4, and thus is rejected under same rationale.
Claim 29 is similar in scope as Claim 5, and thus is rejected under same rationale.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Haveman et al (US20170200302) in view of Scranton et al, Maher et al further in view of Kaizen (“Basic Car Animation in Blender”, 2021).
Regarding Claim 9. The combination of Haveman, Scranton and Maher fails to explicitly teach, however, Kaizen teaches The system of claim 8 wherein the second asset is a vehicle and the configuration defines: a vehicle feature, vehicle element position or combinations thereof (Kaizen, The video describes how to create 3D car animation using Blender®.
Page 2-3, user import a 3D car model to the GUI application.
Page 4, a 3D road background is created.
Page 5-6, a 3D tunnel background is created on top of the road background.
Page 7, the final rendering car animation.).
Haveman, Yasmina, Cheng and Kaizen are analogous art, because they all teach method of generating virtual image by selecting background image and foreground object. Kaizen further teaches creating 3D car animation. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the virtual image generation method (taught by Haveman, Scranton and Maher), to further use Blender® to create 3D car animation (taught in Kaizen), so as to create vehicle video such as car commercial.
Claims 13 are rejected under 35 U.S.C. 103 as being unpatentable over Haveman et al (US20170200302) in view of Scranton et al, Maher et al further in view of Sheeler et al (US20120210262).
Regarding Claim 13. The combination of Haveman, Scranton and Maher fails to explicitly teach, however, Sheeler teaches The system of claim 1 wherein the user interface allows for rendering of the CGI asset in a user specified quality, resolution, aspect ratio or combinations thereof (Sheeler, abstract, the invention describes a media editing application that enables an author-user to create a rig graphically. A rig includes a group of snapshots of one or more parameters at different instances of time. A rig is created by selecting one or more objects and creating snapshots of one or more parameters of the selected objects to create an effect. In some embodiments animation is added to some of the snapshots. Some embodiments provide an edit mode where all parameters that are changed during the edit mode are automatically added to the current snapshot at the end of editing mode.
[0008] Each snapshot in a rig can have an animation (or animated object). That is, the representative parameter of the rig is associated with the properties of the animation. In some embodiments, values for different display aspect ratios are converted into snapshots in a rig. The user by choosing a snapshot can modify the aspect ratio parameters and change the aspect ratio of the video.).
Haveman, Scranton, Maher and Sheeler are analogous art, because they all teach method of generating virtual image based on user’s selecting/configuring virtual object. Sheeler further teaches modifying video based on adjusting parameters including aspect ratio. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the virtual image generation method (taught by Haveman, Scranton and Maher), to further use video parameter adjusting method (taught in Sheeler), so as to provide an intuitive GUI for user to change the setting for the video (Sheeler, [0004]).
Claims 16, 21, 23 are rejected under 35 U.S.C. 103 as being unpatentable over Haveman et al (US20170200302) in view of Scranton et al, Maher et al, Wang et al (US20190082118) further in view of Chapman et al (US12094072).
Regarding Claim 16. The combination of Haveman, Scranton, Maher and Wang further teaches The system of claim 1 further comprising a second software executing on the processor or a second processor, wherein the second software is a CGI software which accesses a storage containing the at least two assets, at least one of said at least two assets is a three dimensional model and (Wang, [0042] FIGS. 3C and 3D illustrate graphical user interfaces with different background scenes selected and showing a recording view and full-screen playback view, according to an embodiment. In FIG. 3C, a recording view is shown where user 302c has selected a virtual background 303c.
[0069] Process 900 continues by receiving a virtual background content (903) from storage. For example, the virtual background content can be a 2D image, 3D image or 360° video. The virtual background content can be selected by the user through a GUI. The virtual background content can be extracted or sampled from any desired virtual environment, such as a famous city or cartoon environment with animated cartoon characters and objects.
Haveman, [0016] FIGS. 11A-11E illustrate a display of 360° content and examples of adjustments to texture, reflection and refraction based on user-selected patterns, orientations and backgrounds, in accordance with the present invention.)
The combination of Haveman, Scranton, Maher and Wang fails to explicitly teach, however, Chapman teaches the CGI software manipulates the three dimensional model based on instructions from the software which are generated in response to user commands input via the user interface (Chapman, abstract, the invention describes Controllable three-dimensional (3D) virtual dioramas in a rendered 3D environment such as a virtual reality or augmented reality environment including one or more rendered objects. 3D diorama is associated with a spatial computing content item such as a downloadable application executable by a computing device. 3D diorama assets may include visual and/or audio content and are presented with rendered 3D environment objects in a composite view, which is presented to a user through a display of computing device. 3D diorama is rotatable in composite view, and at least one 3D diorama asset at least partially occludes, or is at least partially occluded by, at least one rendered 3D environment object. 3D diorama may depict or provide a preview of a spatial computing user experience generated by the downloadable application.
Col 13, line 28-55, 3D diorama 150 can be shown in browse and/or detail pages of portal 120, and assets 152 included in 3D diorama 150 may occlude 190 objects presented in the rendered 3D environment 160, as the assets 152 move in their animation and/or as the scene is rotated 153. For example, if astronaut asset 711 of FIG. 7 moves in its animation to be in front of sofa object 703 of rendered living room, astronaut asset 711 may occlude 190 the view of at least a portion of sofa object 35 703.).
Haveman, Scranton, Maher, Wang and Chapman are analogous art, because they all teach method of generating virtual image based on user’s selecting/configuring virtual object. Chapman further teaches GUI for manipulating the 3D virtual model(s). Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the virtual image generation method (taught by Haveman, Scranton, Maher and Wang), to further use the GUI for modifying 3D virtual model(s) (taught in Chapman), so as to provide user with a collaborative 3D virtual environment (Chapman, col 1, line 21-55).
Claim 21 is similar in scope as Claim 16, and thus is rejected under same rationale.
Claim 23 is similar in scope as Claim 1&16, and thus is rejected under same rationale.
Claims 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Haveman et al (US20170200302) in view of Scranton et al, Maher et al further in Wang et al (US20190082118).
Claim 19 is similar in scope as Claim 11 & 14, and thus is rejected under same rationale.
Claim 20 is similar in scope as Claim 11, and thus is rejected under same rationale.
Claim 24 is rejected under 35 U.S.C. 103 as being unpatentable over Haveman et al (US20170200302) in view of Scranton et al, Maher et al, Wang et al (US20190082118), Chapman et al (US12094072) further in view of Kaizen (“Basic Car Animation in Blender”, 2021).
Regarding Claim 24. The combination of Haveman, Scranton, Maher, Wang and Chapman fails to explicitly teach, however, Kaizen teaches The system of claim 23 wherein the second asset is a three dimensional background and the perspective information is user modifiable via controls on the user interface (Kaizen, The video describes how to create 3D car animation using Blender®.
Page 2-3, user import a 3D car model to the GUI application.
Page 4, a 3D road background is created.
Page 5-6, a 3D tunnel background is created on top of the road background.
Page 7, the final rendering car animation.
From the video, it is clear that user can manually change the camera viewing angle when editing/rendering the car animation.).
Haveman, Scranton, Maher, Wang, Chapman and Kaizen are analogous art, because they all teach method of generating virtual image by selecting background image and foreground object. Kaizen further teaches creating 3D car animation. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the virtual image generation method (taught by Haveman, Scranton, Maher, Wang and Chapman), to further use Blender® to create 3D car animation (taught in Kaizen), so as to create vehicle video such as car commercial.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Steinbach et al (US10460497), abstract, the invention describes methods for generating content (e.g., a feature film, virtual reality experience) in a virtual environment (e.g., a VR environment). Specifically, they allow fast prototyping and development of a virtual reality experience by allowing virtual assets to be quickly imported into a virtual environment. The virtual assets can be used to help visualize or
"storyboard" an item of content during early stages of development. In doing so, the content can be rapidly iterated upon without requiring use of more substantial assets, which can be time consuming and resource intensive.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIN SHENG whose telephone number is (571)272-5734. The examiner can normally be reached M-F 9:30AM-3:30PM 6:00PM-8:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 5712723022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Xin Sheng/Primary Examiner, Art Unit 2619
Prosecution Insights