DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
.Response to Amendment
This Office Action is in response to Applicant’s RCE amendment filed 05/08/2026 which has been entered and made of record. Claims 1, 3, 6-7, 14 and 19 have been amended. Claims 16-17 have been cancelled. Claims 21-22 have been newly added. Claims 1-15 and 18-22 are pending in the application.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 7 and 14 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument (due to applicant’s arguments directed to newly amend limitation(s) which is addressed by new prior art presented in this Office Action).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5, 7-9, 11-14, 19-20 and 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Reichow et al. (U.S. Patent Application Publication No. 2018/0122122), hereinafter referenced as Reichow, in view Russell et al. (U.S. Patent Application Publication No. 2022/0327754), hereinafter referenced as Russell, and Eck et al. (U.S. Patent Application Publication No. 2022/0405997), hereinafter referenced as Eck.
Regarding claim 1, Reichow teaches a method comprising: determining an orientation of a mechanically animated surface relative to one or more projectors configured to project content onto the mechanically animated surface (paragraph 38 teaches "techniques of using projection to augment mechanically animated objects...FIG. 3A shows a portion of a display system 300 that includes a screen element 310 that includes a projection surface 312 useful for front projection from a video projector 330 (not projecting in FIG. 3A). The screen element 310 may be thought of as including a substantially planar body...with several components or members that can be actuated to cause the projection surface 312 to move in location and/or morph in shape or size." and paragraph 39 teaches "the screen element 310 is positioned within a display space at a location and with an orientation suited to a location at which a person was videotaped in that same display space (or a location matching where a CG animation was created for insertion into that display space)."); fig. 3A visually supports this description and shows screen element 310 is mechanically animated surface and is placed within display space relative to projector 330 (which in later steps and figures shows projecting content onto screen element 310/mechanically animated surface) and after orientation is determined; dynamically rendering the content based on one or more parameters derived from the change in the orientation of the mechanically animated surface relative to the one or more projectors (paragraph 10 teaches "screen is used to provide a character...responses a rigged, real-time rendered facial response is provided via projection onto the screen.", paragraph 40 teaches "FIG. 3B, the actuator assembly has been operated to actuate the screen element 310 (e.g., to provide mechanical animation 324, 326 to the element 310). As a result, the projection surface 312A is in a new state with the arm 323 raised...mechanical animation or actuation by the actuator assembly has morphed or modified the projection surface 312A" and paragraph 41 teaches "projector 330 is being operated to project content/light 333 onto this projection surface 312A"); this and fig. 3B explicitly shows rendering occurs dynamically (according to movement) and that the rendering is based on a parameter of location/orientation of mechanically animated surface/projection surface 312a relative to the projector which projects; and presenting the content on the mechanically animated surface using the one or more projectors (paragraph 41 teaches "causes the image of a character (e.g., a person) to be reflected from the surface 312A of the screen element 310 to a viewer of the display space in which the screen element 310 is provided/positioned. In practice, the screen actuator provides slow and gross animation while the video content 333 providing the animated imagery 335 provides high frequency animation or augmentation. Particularly, the character's face is provided in great detail with animation provided as shown with arrows 337, 338, 339 corresponding with the character talking, moving their eyes, and having their hair flow/move in response to airflow in the display space (e.g., provided by a fan in a contextual display assembly). Further, the character provided with reflected content 335 is wearing clothes that are shown to move 336 in response to airflow in the display space (or movement of the screen element 310)"); this shows content presented (by projector) on mechanically animated surface/ surface 312A.
However, Reichow fails to explicitly teach determining a change in the orientation of the mechanically animated surface relative to the one or more projectors; dynamically rendering…, wherein the content is responsive to the change in the orientation of the mechanically animated surface.
However, Russell explicitly teaches determining a change in the orientation of the mechanically animated surface relative to the one or more projectors (Russell, paragraph 19 teaches “Instead, the animated figure display system of the present disclosure may be fitted with trackers that enable tracking cameras to discern movements, positions, and orientations of projection surfaces (e.g., an animated figure) in real-time”); this shows tracking/determining change in orientation of the mechanically animated surface/figure which is relative to projector as shown in figs. 3-7; dynamically rendering…, wherein the content is responsive to the change in the orientation of the mechanically animated surface (Russell, paragraph 20 teaches “to enhance the authenticity of the animated figure, the animated figure and/or any number of projection surfaces may be fitted with trackers that enable tracking cameras of a tracking and media control system to discern movements, positions, and orientations of the animated…the tracking and media control system may dynamically generate and display projected images onto the interactive animated figure and/or the projection surfaces that emulates live characters, movement, and/or reaction to other effects”); this explicitly shows the rendering would be dynamic and that the content generated is responsive to the change in orientation. Russell is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of dynamic rendering and content projected on animatronic in response to orientation/position. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Reichow's invention with the dynamic content techniques of Russell to provide immersive viewing experiences for guests, but without the challenges and/or costs associated with providing such experiences using projection mapping (Russell, paragraph 19). This means higher user/guest satisfaction.
However, the combination of Reichow and Russell fails to explicitly teach and a real-time relative positioning of the one or more projectors with respect to the mechanically animated surface. Although, Russell, paragraph 20 teaches "the tracking control system may utilize the location data to control any number of projectors" and paragraph 22 teaches
"control operation of the projectors 112 based on location data" (both of which show the projector being controlled); which implies to one of ordinary skill in the art, alongside the above paragraph 19 citations, that relative positioning of the projectors (to the surface) would be updated to accurately project onto a moving/changing surface in real-time.
However, Eck explicitly teaches and a real-time relative positioning of the one or more projectors with respect to the mechanically animated surface (Eck, paragraph 41 teaches “initial projection and/or tracking design may simulate an appearance (e.g., from the guests' perspective) of imagery output via a projector onto the character from a particular positioning (e.g., location, orientation) of the projector…projection study may be used to determine whether the image data used to output the imagery is to be adjusted, the positioning of the projector is to be changed” and paragraph 57 teaches “identifying markers positioned at the animated figure and synchronizing projections with movement of the animated figure along with adjustment and tuning of the projectors and projections”); since synchronized with movement, this is done in real time, and this shows relative positioning of the projector(s) with respect to mechanically animated surface being updated since the projector positioning is adjusted due to such. Eck is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of projection of imagery onto a tangible model which can be updated/have movement. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Reichow and Russell with the motor and specialized technology of animatronic techniques of Eck so numerous and variable iteration routines are included to gradually improve aspects of the production and are repeated until an approved level of realism or immersion is achieved (Eck, paragraph 22). This would mean more realistic animatronic and better user experience.
Regarding claim 2, the combination of Reichow, Russell and Eck teaches further comprising dynamically generating one or more movements of the mechanically animated surface, (Reichow, paragraph 5 teaches "a display system (and associated methods provided by its operations) is provided that is configured to provide projection mapped augmentation of mechanically animated objects. These mechanically animated objects may take the form of a screen assembly with a 2D or 3D projection surface(s), and a screen actuator may be included in the display system that is operated by a controller to move, to expand, or other otherwise operate the screen assembly so as to modify or morph the projection surface to move and/or change its shape and/or size." ); this shows dynamic movement of the display screen assembly/mechanically animated surface; wherein the one or more movements of the mechanically animated surface are synchronized to the content (Reichow, paragraph 5 teaches "The display system further includes a projector(s) operated by the controller (in a synchronized manner with the screen actuator) to project content onto the projection surface as it is moved and/or has its shape and/or size changed over time. The projected content is mapped to the various states and movement of the projection surface"); this shows projector/content projection synced with movement of actuator (part of the mechanically animated surface).
Regarding claim 3, the combination of Reichow, Russell and Eck teaches further comprising directly controlling one or more motors configured to actuate the mechanically animated surface to cause the mechanically animated surface to make the one or more movements during the presenting of the content on the mechanically animated surface (Eck, paragraph 50 teaches "as represented by block 130, may include incorporation of servo-motors and a controller for automatically synchronizing motion of relevant actuations. For example, actuators associated with facial features may be automatically operated (e.g., without a user manually operating each individual actuator) in various synched movements to display an array of facial expressions on a fabricated head and face with flexible skin."); these servo-motors are controlled/used for controlling and syncing movement of an animatronic which would happen during presentation of content on the mechanically animated surface from Reichow. The same motivations used in claim 1 apply here in claim 3.
Regarding claim 5, the combination of Reichow, Russell and Eck teaches wherein the orientation of the mechanically animated surface relative to the one or more projectors is determined based on input from one or more animatronic sensors associated with the mechanically animated surface and the one or more projectors (Reichow, paragraph 10 teaches "The media/animation (or media content) can thought of as being “rigged” so that as the mechanical screen surface moves (such as in response to a viewer's interactions) the media can be real-time generated to fit the surface of the screen and its surface(s). This provides a visual effect that is similar to that provided with a robotic interactive character responding to nearby people's gestures and actions. The visual and audio sensor sense what the people just did and provides this information to a system controller that uses its programming/software (e.g., a game engine or the like) to respond with an understanding of a proper response (movement of the screen and/or change in media content that may be generated in real time)." and
paragraph 39 teaches "the screen element 310 is positioned within a display space at a location and with an orientation suited to a location at which a person was videotaped in that same display space (or a location matching where a CG animation was created for insertion into that display space)."); visual and audio sensor and responding with proper response/movement shows input received from animatronic sensor associated with the animatronic, further this shows location/position derived for screen element 310/mechanical animated surface and this is suited to a location at which a person was videotaped in that same display space meaning it is from the visual sensor input.
Regarding claim 7, Reichow teaches a method comprising: receiving input from at least one animatronic sensor associated with an animatronic, (paragraph 10 teaches "The media/animation (or media content) can thought of as being “rigged” so that as the mechanical screen surface moves (such as in response to a viewer's interactions) the media can be real-time generated to fit the surface of the screen and its surface(s). This provides a visual effect that is similar to that provided with a robotic interactive character responding to nearby people's gestures and actions. The visual and audio sensor sense what the people just did and provides this information to a system controller that uses its programming/software (e.g., a game engine or the like) to respond with an understanding of a proper response (movement of the screen and/or change in media content that may be generated in real time)."); visual and audio sensor and responding with proper response/movement shows input received from animatronic sensor associated with the animatronic; the animatronic including a mechanically animated surface (paragraph 5 teaches "a display system (and associated methods provided by its operations) is provided that is configured to provide projection mapped augmentation of mechanically animated objects. These mechanically animated objects may take the form of a screen assembly with a 2D or 3D projection surface(s), and a screen actuator may be included in the display system that is operated by a controller to move, to expand, or other otherwise operate the screen assembly so as to modify or morph the projection surface to move and/or change its shape and/or size" and fig. 3A, reference 310 shows the screen element/mechanically animated surface); mechanically animated objects taking form of screen assembly and actuator included in display system shows the animatronic depicted includes mechanically animated surface; deriving positional information for the mechanically animated surface from the input (paragraph 39 teaches "the screen element 310 is positioned within a display space at a location and with an orientation suited to a location at which a person was videotaped in that same display space (or a location matching where a CG animation was created for insertion into that display space)."); this shows location/position derived for screen element 310/mechanical animated surface and this is suited to a location at which a person was videotaped in that same display space meaning it is from the visual sensor input; dynamically rendering a next frame of the content based on: the change in the positional information (paragraph 42 teaches "In FIG. 3C, the screen actuator has further operated to return the arm 323 to its down or original position of FIG. 3A, to move the leg 325 back away from the other stationary leg, and to tilt or rotate the head 321 downward to one side. In this way, the screen element 310 is in a new state such that projection surface 312B differs from projection surface 335 in shape (or location of portions such as head 321, arm 323, and leg 325). The content/light 335 being projected is chosen (by a controller) to differ, too,"); content being projected differing due to location shows rendering of a next frame of content that is projected onto the mechanically animated surface and this would be based on positional information since it's after and due to the movement and/or tilt and/or rotation of location; and providing the next frame of content to the at least one projector (paragraph 42 teaches "The content/light 335 being projected is chosen (by a controller) to differ, too, in its mapping (i.e., mapped to new silhouette shape/location) and also what is being displayed as the animated character (e.g., reflected content 337) reflected from the surface 312B has changed (e.g., a different facial expression, different movement/location their eyes/pupils, and so on)."); this alongside fig. 3C shows projection of next frame of content from projector to mechanically animated surface, thus next frame of content must be provided to projector.
However, Reichow fails to explicitly teach determining a change in the positional information; dynamically rendering…, wherein the next frame of content is responsive to the change in the positional information.
However, Russell explicitly teaches determining a change in the positional information (Russell, paragraph 19 teaches “Instead, the animated figure display system of the present disclosure may be fitted with trackers that enable tracking cameras to discern movements, positions, and orientations of projection surfaces (e.g., an animated figure) in real-time”); this shows tracking/determining change in position information of the mechanically animated surface/figure which is relative to projector as shown in figs. 3-7; dynamically rendering…, wherein the next frame of content is responsive to the change in the positional information (Russell, paragraph 20 teaches “to enhance the authenticity of the animated figure, the animated figure and/or any number of projection surfaces may be fitted with trackers that enable tracking cameras of a tracking and media control system to discern movements, positions, and orientations of the animated…the tracking and media control system may dynamically generate and display projected images onto the interactive animated figure and/or the projection surfaces that emulates live characters, movement, and/or reaction to other effects”); this explicitly shows the rendering would be dynamic and that the content generated is responsive to the change in position. Russell is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of dynamic rendering and content projected on animatronic in response to orientation/position. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Reichow's invention with the dynamic content techniques of Russell to provide immersive viewing experiences for guests, but without the challenges and/or costs associated with providing such experiences using projection mapping (Russell, paragraph 19). This means higher user/guest satisfaction.
However, the combination of Reichow and Russell fails to explicitly teach
determining a change in the positional information relative to at least one projector projecting content onto the mechanically animated surface; and a real-time relative positioning of the at least one projector with respect to the mechanically animated surface.
However, Eck explicitly teaches determining a change in the positional information relative to at least one projector projecting content onto the mechanically animated surface (Eck, paragraph 41 teaches “initial projection and/or tracking design may simulate an appearance (e.g., from the guests' perspective) of imagery output via a projector onto the character from a particular positioning (e.g., location, orientation) of the projector…projection study may be used to determine whether the image data used to output the imagery is to be adjusted, the positioning of the projector is to be changed”); tracking design simulating appearance from projector onto character/surface and projection study here show that change in positional information is tracked/determined, which is relative to projector(s) projecting content onto surface since it is determined using the image data (on surface) to see whether projection positioning is to be changed; and a real-time relative positioning of the at least one projector with respect to the mechanically animated surface (Eck, paragraph 41 teaches “initial projection and/or tracking design may simulate an appearance (e.g., from the guests' perspective) of imagery output via a projector onto the character from a particular positioning (e.g., location, orientation) of the projector…projection study may be used to determine whether the image data used to output the imagery is to be adjusted, the positioning of the projector is to be changed” and paragraph 57 teaches “identifying markers positioned at the animated figure and synchronizing projections with movement of the animated figure along with adjustment and tuning of the projectors and projections”); since synchronized with movement, this is done in real time, and this shows relative positioning of the projector(s) with respect to mechanically animated surface being updated since the projector positioning is adjusted due to such.
Eck is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of projection of imagery onto a tangible model which can be updated/have movement. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Reichow and Russell with the motor and specialized technology of animatronic techniques of Eck so numerous and variable iteration routines are included to gradually improve aspects of the production and are repeated until an approved level of realism or immersion is achieved (Eck, paragraph 22). This would mean more realistic animatronic and better user experience.
Regarding claim 8, the combination of Reichow, Russell and Eck teaches further comprising dynamically generating one or more movements of the mechanically animated surface based on the positional information, (Reichow, paragraph 5 teaches "a display system (and associated methods provided by its operations) is provided that is configured to provide projection mapped augmentation of mechanically animated objects. These mechanically animated objects may take the form of a screen assembly with a 2D or 3D projection surface(s), and a screen actuator may be included in the display system that is operated by a controller to move, to expand, or other otherwise operate the screen assembly so as to modify or morph the projection surface to move and/or change its shape and/or size." and paragraph 20 teaches "screen assembly is adapted to be moved or operated by the screen actuator to provide a changing projection surface(s) with a shape, size, and location that changes over time to match the choreographed dance" ); this shows dynamic movement of the display screen assembly/mechanically animated surface and changing projection surface with location change over time explicitly shows this is based on positional information; wherein the one or more movements of the mechanically animated surface are synchronized to the next frame of content (Reichow, paragraph 5 teaches "The display system further includes a projector(s) operated by the controller (in a synchronized manner with the screen actuator) to project content onto the projection surface as it is moved and/or has its shape and/or size changed over time. The projected content is mapped to the various states and movement of the projection surface"); this shows projector/content projection synced with movement of actuator (part of the mechanically animated surface) meaning next frame displayed by projector would also be synchronized to the movements.
Regarding claim 9, the combination of Reichow, Russell and Eck teaches further comprising providing the one or more movements of the mechanically animated surface to a control system of the animatronic (Reichow, paragraph 19 teaches "screen actuator is included in the system that is operated by a system controller to morph/change in shape and/or size the projection surfaces (e.g., in 2D or 3D) and/or to move or be animated (again, in 2D or 3D)"); this shows system controller used to change/morph and adjust/move the projection surfaces/mechanically animated surface meaning the movements must be provided to it.
Regarding claim 11, the combination of Reichow, Russell and Eck teaches wherein the at least one animatronic sensor is a camera (Eck, paragraph 61 teaches "with respect to completion of the process 220, the media control system of certain embodiments may generate and update a skeletal model of the animated figure based on feedback from the tracking cameras. The skeletal model generally represents the moveable portions of the animated figure, such as actuatable joints thereof, and is dynamically updated to represent a current position (e.g., including x, y, and z translational coordinates and/or x, y, z rotational coordinates), orientation, and/or scale of the animated figure or portions thereof (e.g., a pose of the animated figure)"); this explicitly shows camera used to input/update information for animatronic making it act as an animatronic sensor. The same motivations used in claim 1 apply here in claim 11.
Regarding claim 12, the combination of Reichow, Russell and Eck teaches further comprising receiving additional input from the at least one animatronic sensor as the content is projected onto the mechanically animated surface (Eck paragraph 61 teaches "with respect to completion of the process 220, the media control system of certain embodiments may generate and update a skeletal model of the animated figure based on feedback from the tracking cameras. The skeletal model generally represents the moveable portions of the animated figure, such as actuatable joints thereof, and is dynamically updated to represent a current position (e.g., including x, y, and z translational coordinates and/or x, y, z rotational coordinates), orientation, and/or scale of the animated figure or portions thereof (e.g., a pose of the animated figure)"); feedback from tracking camera is additional input from sensor and since model/animatronic is dynamically updated to represent a current position, it is updated as the content is projected onto the mechanically animated surface. The same motivations used in claim 3 apply here in claim 12.
Regarding claim 13, the combination of Reichow, Russell and Eck teaches wherein the next frame of content is dynamically rendered using a game engine (Reichow, paragraph 10 teaches "The visual and audio sensor sense what the people just did and provides this information to a system controller that uses its programming/software (e.g., a game engine or the like) to respond with an understanding of a proper response (movement of the screen and/or change in media content that may be generated in real time)."); this shows game engine used for the rendering.
Regarding claim 14, Reichow teaches a system comprising: one or more projectors (fig. 3 shows projector 330, fig. 1 and abstract teach "the display system further includes a projector"); an animatronic comprising a mechanically animated surface (paragraph 5 teaches "a display system (and associated methods provided by its operations) is provided that is configured to provide projection mapped augmentation of mechanically animated objects. These mechanically animated objects may take the form of a screen assembly with a 2D or 3D projection surface(s), and a screen actuator may be included in the display system that is operated by a controller to move, to expand, or other otherwise operate the screen assembly so as to modify or morph the projection surface to move and/or change its shape and/or size" and fig. 3A, reference 310 shows the screen element/mechanically animated surface); mechanically animated objects taking form of screen assembly and actuator included in display system shows the animatronic depicted includes mechanically animated surface; and a performance system in communication with the one or more projectors and the animatronic, (paragraph 31 teaches "display controller 140 uses the processor 142 to execute code in computer readable media (e.g., to run a software program) to provide the control functions described herein. This is shown with the CPU 142 running a display control module (e.g., a “projectormatronics” program) 146. The display control module 146 is configured to generate and transmit control signals 160 to the projector 130 to cause it to project the media content/light 135 onto the projection surface 117A-117C. Further, the display control module 146 acts to generate and transmit control signals 162 to the screen actuator 118 to cause it to actuate 119A and 119B the screen element 116A-116C" ); display controller acts as performance system since it comprises display module which communicates between projector and animatronic using control signals; the performance system being configured to: and dynamically render a projection of content from the one or more projectors onto the mechanically animated surface (paragraph 10 teaches "a system controller that uses its programming/software (e.g., a game engine or the like) to respond with an understanding of a proper response (movement of the screen and/or change in media content that may be generated in real time). screen is used to provide a character...responses a rigged, real-time rendered facial response is provided via projection onto the screen.", paragraph 40 teaches "FIG. 3B, the actuator assembly has been operated to actuate the screen element 310 (e.g., to provide mechanical animation 324, 326 to the element 310). As a result, the projection surface 312A is in a new state with the arm 323 raised...mechanical animation or actuation by the actuator assembly has morphed or modified the projection surface 312A" and paragraph 41 teaches "projector 330 is being operated to project content/light 333 onto this projection surface 312A"); controller/performance system does this and fig. 3B explicitly shows rendering occurs dynamically (according to movement) and that the rendering is based on a parameter of location/orientation of mechanically animated surface/projection surface 312a relative to the projector which projects; wherein the dynamically rendered projection of content is synchronized with the one or more movements of the mechanically animated surface (paragraph 5 teaches "The display system further includes a projector(s) operated by the controller (in a synchronized manner with the screen actuator) to project content onto the projection surface as it is moved and/or has its shape and/or size changed over time. The projected content is mapped to the various states and movement of the projection surface"); this shows projector/content projection synced with movement of actuator (part of the mechanically animated surface).
However, Reichow fails to explicitly teach determine one or more movements of the mechanically animated surface; dynamically rendering…, and wherein the dynamically rendered projection of content is responsive to the one or more movements of the mechanically animated surface.
However, Russell explicitly teaches determine one or more movements of the mechanically animated surface (Russell, paragraph 19 teaches “Instead, the animated figure display system of the present disclosure may be fitted with trackers that enable tracking cameras to discern movements, positions, and orientations of projection surfaces (e.g., an animated figure) in real-time”); this shows tracking change in position information (determining movement) of the mechanically animated surface/figure which is relative to projector as shown in figs. 3-7; dynamically rendering…, and wherein the dynamically rendered projection of content is responsive to the one or more movements of the mechanically animated surface (Russell, paragraph 20 teaches “to enhance the authenticity of the animated figure, the animated figure and/or any number of projection surfaces may be fitted with trackers that enable tracking cameras of a tracking and media control system to discern movements, positions, and orientations of the animated…the tracking and media control system may dynamically generate and display projected images onto the interactive animated figure and/or the projection surfaces that emulates live characters, movement, and/or reaction to other effects”); this explicitly shows the rendering would be dynamic and that the content generated is responsive to the movements. Russell is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of dynamic rendering and content projected on animatronic in response to orientation/position. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Reichow's invention with the dynamic content techniques of Russell to provide immersive viewing experiences for guests, but without the challenges and/or costs associated with providing such experiences using projection mapping (Russell, paragraph 19). This means higher user/guest satisfaction.
However, the combination of Reichow and Russell fails to explicitly teach
determining a change in the positional information determine a real-time orientation of the mechanically animated surface relative to the one or more projectors; Although, Reichow, paragraph 38 teaches "techniques of using projection to augment mechanically animated objects...FIG. 3A shows a portion of a display system 300 that includes a screen element 310 that includes a projection surface 312 useful for front projection from a video projector 330 (not projecting in FIG. 3A). The screen element 310 may be thought of as including a substantially planar body...with several components or members that can be actuated to cause the projection surface 312 to move in location and/or morph in shape or size." and paragraph 39 teaches "the screen element 310 is positioned within a display space at a location and with an orientation suited to a location at which a person was videotaped in that same display space (or a location matching where a CG animation was created for insertion into that display space)."); fig. 3A visually supports this description and shows screen element 310 is mechanically animated surface and is placed within display space relative to projector 330 (which in later steps and figures shows projecting content onto screen element 310/mechanically animated surface) and after orientation is determined;
dynamically render… and based on the real-time orientation of the mechanically animated surface relative to the one or more projectors. Although, Reichow, paragraph 10 teaches "screen is used to provide a character...responses a rigged, real-time rendered facial response is provided via projection onto the screen.", paragraph 40 teaches "FIG. 3B, the actuator assembly has been operated to actuate the screen element 310 (e.g., to provide mechanical animation 324, 326 to the element 310). As a result, the projection surface 312A is in a new state with the arm 323 raised...mechanical animation or actuation by the actuator assembly has morphed or modified the projection surface 312A" and paragraph 41 teaches "projector 330 is being operated to project content/light 333 onto this projection surface 312A"); this and fig. 3B explicitly shows rendering occurs dynamically (according to movement) and that the rendering is based on a parameter of location/orientation of mechanically animated surface/projection surface 312a relative to the projector which projects.
However, Eck explicitly teaches determine a real-time orientation of the mechanically animated surface relative to the one or more projectors (Eck, paragraph 41 teaches “initial projection and/or tracking design may simulate an appearance (e.g., from the guests' perspective) of imagery output via a projector onto the character from a particular positioning (e.g., location, orientation) of the projector…projection study may be used to determine whether the image data used to output the imagery is to be adjusted, the positioning of the projector is to be changed”, and paragraph 58 teaches “animated figure may be fitted with trackers that enable tracking cameras of a media control system to discern movements, positions, and orientations of the animated figure in real-time”); tracking design simulating appearance from projector onto character/surface (from particular orientation) and projection study here show that change in orientation information is tracked/determined, as mentioned this is done in real time for orientation of animated figure/surface, and this is also of surface relative to projector(s) since it is determined using the image data (on surface) to see whether projection positioning/orientation is to be changed;
dynamically render… and based on the real-time orientation of the mechanically animated surface relative to the one or more projectors.
(Eck, paragraph 57 teaches “identifying markers positioned at the animated figure and synchronizing projections with movement of the animated figure along with adjustment and tuning of the projectors and projections” and paragraph 58 teaches “animated figure may be fitted with trackers that enable tracking cameras of a media control system to discern movements, positions, and orientations of the animated figure in real-time”); this shows that the dynamic rendering from the combination above would be based on real-time orientation of animated figure/surface which as aforementioned is relative to projector(s).
Eck is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of projection of imagery onto a tangible model which can be updated/have movement. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Reichow and Russell with the motor and specialized technology of animatronic techniques of Eck so numerous and variable iteration routines are included to gradually improve aspects of the production and are repeated until an approved level of realism or immersion is achieved (Eck, paragraph 22). This would mean more realistic animatronic and better user experience.
Regarding claim 19, the combination of Reichow, Russell and Eck teaches further comprising one or more animatronic sensors associated with the animatronic, (Reichow, paragraph 10 teaches "The media/animation (or media content) can thought of as being “rigged” so that as the mechanical screen surface moves (such as in response to a viewer's interactions) the media can be real-time generated to fit the surface of the screen and its surface(s). This provides a visual effect that is similar to that provided with a robotic interactive character responding to nearby people's gestures and actions. The visual and audio sensor sense what the people just did and provides this information to a system controller that uses its programming/software (e.g., a game engine or the like) to respond with an understanding of a proper response (movement of the screen and/or change in media content that may be generated in real time)."); visual and audio sensor and responding with proper response/movement shows input received from animatronic sensor associated with the animatronic; wherein the performance system is further configured to derive positional information for the mechanically animated surface from output of the one or more animatronic sensors (Reichow, paragraph 39 teaches "the screen element 310 is positioned within a display space at a location and with an orientation suited to a location at which a person was videotaped in that same display space (or a location matching where a CG animation was created for insertion into that display space)."); this shows location/position derived for screen element 310/mechanical animated surface and this is suited to a location at which a person was videotaped in that same display space meaning it is from the visual sensor output; and dynamically render a next frame of the projection of content based on the positional information (Reichow, paragraph 42 teaches "In FIG. 3C, the screen actuator has further operated to return the arm 323 to its down or original position of FIG. 3A, to move the leg 325 back away from the other stationary leg, and to tilt or rotate the head 321 downward to one side. In this way, the screen element 310 is in a new state such that projection surface 312B differs from projection surface 335 in shape (or location of portions such as head 321, arm 323, and leg 325). The content/light 335 being projected is chosen (by a controller) to differ, too,"); content being projected differing due to location shows rendering of a next frame of content that is projected onto the mechanically animated surface and this would be based on positional information since it's after and due to the movement and/or tilt and/or rotation of location.
Regarding claim 20, the combination of Reichow, Russell and Eck teaches wherein the performance system is further configured to provide the next frame of content to the one or more projectors (Reichow, paragraph 42 teaches "The content/light 335 being projected is chosen (by a controller) to differ, too, in its mapping (i.e., mapped to new silhouette shape/location) and also what is being displayed as the animated character (e.g., reflected content 337) reflected from the surface 312B has changed (e.g., a different facial expression, different movement/location their eyes/pupils, and so on)."); this alongside fig. 3C shows projection of next frame of content from projector (meaning next frame must be provided to projector) to mechanically animated surface.
Regarding claim 22, the combination of Reichow, Russell and Eck wherein dynamically rendering the content further comprises analyzing, using an artificial intelligence model, the one or more parameters and the real-time relative positioning of the one or more projectors to dynamically render the content (Eck, paragraph 48 teaches “assessment of the 3D character model and the projected imagery may be based on an algorithm, artificial intelligence, user input, and/or the like” and paragraph 22 teaches “ Due to the complex nature of such interactions, in some embodiments, control instructions from an artificial intelligence (AI) or machine learning engine may coordinate media, projection location, animation speed, projection surface positioning, and the like to create a realistic and immersive experience”); since AI is used for projection location, positioning and the like, alongside the assessment/analyzing, this shows AI analyzing one or more parameters and real-time relative positioning of the projector(s) by AI to dynamically render content. The same motivations used in claim 1 apply here in claim 22.
Claim(s) 4 and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Reichow, Russell and Eck as applied to claims 1 and 7 above, and further in view of Grundhofer et al. (U.S. Patent Application Publication No. 2015/0154783), hereinafter referenced as Grundhofer.
Regarding claim 4, the combination of Reichow, Russell and Eck fails to explicitly teach wherein the content is dynamically rendered using vertex data and animation data.
However, Grundhofer explicitly teaches wherein the content is dynamically rendered using vertex data and animation data (Grundhofer, paragraph 2 teaches "Animated animatronic figures, such as avatars, are a unique way to give physical presence to a character. For example, many animatronic figures are movable and can be used as part of an interactive display for people at a theme park", paragraph 4 teaches "augmenting the appearance of an object including a plurality of projectors", and paragraph 80 teaches "the target performance 109 is rendered and the images are deformed to match the physical structure of the avatar 102. The deformation involves the image...deformation helps to adapt the desired features of the target performance 109 to the avatar 102, while also preserving the artistic intent of the target performance 109. This allows a user the means to indicate the semantics of the animation by selecting individual or curves of vertices of the target performance 109 and assign a property to it."); this shows dynamic content using animation data, vertices assigned a property and selected by user to render projection on animatronic. Grundhofer is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of projecting onto movable animatronics and using vertices as well as animation data to do so. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Reichow, Russell and Eck with the vertex and animation data techniques of Grundhofer to improve the optimization process (Grundhofer, paragraph 72). This would be done by user assigning specific properties to vertices to indicate semantics of the animation.
Regarding claim 10, the combination of Reichow, Russell, Eck and Grundhofer teaches wherein the next frame of content is dynamically rendered using vertex data and animation data (Grundhofer, paragraph 2 teaches "Animated animatronic figures, such as avatars, are a unique way to give physical presence to a character. For example, many animatronic figures are movable and can be used as part of an interactive display for people at a theme park", paragraph 4 teaches "augmenting the appearance of an object including a plurality of projectors", and paragraph 80 teaches "the target performance 109 is rendered and the images are deformed to match the physical structure of the avatar 102. The deformation involves the image...deformation helps to adapt the desired features of the target performance 109 to the avatar 102, while also preserving the artistic intent of the target performance 109. This allows a user the means to indicate the semantics of the animation by selecting individual or curves of vertices of the target performance 109 and assign a property to it."); this shows dynamic content (inclusive of next frame from Reichow) using animation data, vertices assigned a property and selected by user to render projection on animatronic. The same motivations used in claim 4 apply here in claim 10.
Claim(s) 6 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Reichow, Russell and Eck as applied to claims 1 and 14 above, and further in view of Lee et al. (U.S. Patent Application Publication No. 2025/0126237), hereinafter referenced as Lee.
Regarding claim 6, the combination of Reichow, Russell and Eck fails to explicitly teach further comprising dynamically rendering the content based on an angle of projection, a ratio of projection or an intensity of projection.
However, Lee explicitly teaches further comprising dynamically rendering the content based on an angle of projection, a ratio of projection or an intensity of projection (Lee, paragraph 154 teaches "the at least one processor 111 may maintain the projection angle or control the angle to the limit angle, and then change the size of the content.", paragraph 182 teaches "The projection part 112 may output an image in a screen ratio of 4:3, a screen ratio of 5:4, and a wide screen ratio of 16:9 according to the use of the electronic device 100 or the user's setting, etc" and paragraph 188 teaches "The projection part 112 may provide a dimming function for adjusting the intensity of a light source to the user"); this shows parameters of angle, ratio and intensity of projection while projecting/dynamically rendering content. Lee is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of using multiple projectors and having parameters thereof such as angle, ratio and intensity of projection. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Reichow, Russell and Eck with the projector parameter techniques of Lee to project contents in the brightness corresponding to the lowest power, contents may be projected during a long time while visibility for the user is improved (Lee, paragraph 444).
Regarding claim 18, the combination of Reichow, Russell, Eck and Lee teaches wherein dynamically rendering the projection of content comprises rendering the projection of content to be presented by two or more projectors (Lee, paragraph 117 teaches "with the multi-view function, two or more projectors may respectively project allotted contents."); this shows for multi-view, rendering being done with multiple projectors. The same motivations used in claim 6 apply here in claim 18.
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Reichow, Russell and Eck as applied to claim 14 above, and further in view of Mecca et al. (U.S. Patent No. 11,772,276), hereinafter referenced as Mecca.
Regarding claim 15, the combination of Reichow, Russell and Eck fails to explicitly teach wherein the performance system comprises a real-time rendering engine in communication with the one or more projectors and a mechanical control system of the animatronic.
However, Mecca explicitly teaches wherein the performance system comprises a real-time rendering engine in communication with the one or more projectors and a mechanical control system of the animatronic (Mecca, col. 9, lines 6-11 teach "the interactive data sources 70 may include a dedicated game engine or computing device that dynamically generates instructions executable by the animated figure controller 102 to cause the animated FIG. 12 to modify its movements within the amusement attraction 10" and col. 10, lines 61-66 teach "media controller 112 may be any suitable media generator or game engine with significant processing power and reduced latency, in accordance with the present disclosure. It should be understood that the media controller 112 is therefore capable of generating the images to be projected onto the animated FIG. 12 in substantially real-time"); game engine here is real-time rendering engine since it renders images in real-time and this game engine causes animated figure to modify movements (of animatronic which is part of motion/mechanical control system per Mecca abstract) as well as generates images projected onto the animated figure. Mecca is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of having an animated figure with a body and actuators configured to adjust a figure portion of the body in response to interactive data received from one or more interactive data sources and generate data indicative of images to be projected onto an external surface of the figure portion having the current position and orientation. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Reichow, Russell and Eck with the rendering engine and communication techniques of Mecca to provide improved show quality with reduced latency and enhanced realism (Mecca, col. 6, lines 45-46). This would be done by having an efficient rendering engine that communicates with projector and animated figure/animatronic.
Claim(s) 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Reichow, Russell, Eck and Grundhofer as applied to claim 4 above, and further in view of Lindauer et al. (U.S. Patent Application Publication No. 2023/0113713), hereinafter referenced as Lindauer.
Regarding claim 21, the combination of Reichow, Russell, Eck and Grundhofer teaches generating, using the vertex data and the animation data, a mesh of the mechanically animated surface (Grundhofer, paragraph 35 teaches “the target performance is mapped to a mesh sequence and the mesh sequence is fitted to the avatar”); this shows mesh of mechanically animated surface/avatar generated by using the aforementioned vertex and animation data mentioned in claim 4 above; wherein dynamically rendering the content is further based on the shape of the mechanically animated surface (Reichow, paragraph 12 teaches “shape and the size of the projection screen…second media content is mapped to the modified shape and size during the projecting of the second media content”); this shows the dynamically rendering content is based on shape of projection screen/mechanically animated surface.
However, the combination of Reichow, Russell, Eck and Grundhofer fails to teach performing a strain analysis on the mesh to determine one or more protruding points in the mesh; and generating a shape of the mechanically animated surface based on the one or more protruding points,
However, Lindauer teaches performing a strain analysis on the mesh to determine one or more protruding points in the mesh (Lindauer, paragraph 54 teaches “the quality of the fit of the mesh may be determined using one or more fit criteria. For example, distortion analysis which measure the amount of distortion of individual or groups of mesh elements, may be used to determine if fitting the face foam to the face results in unacceptable amount (e.g., greater than 5%) distortion of the face foam mesh.”); distortion analysis shows strain analysis and is on mesh to determine a percent/amount of distortion which when above a certain percent like 5% mentioned would be considered protruding point(s) of mesh; and generating a shape of the mechanically animated surface based on the one or more protruding points, (Lindauer, paragraph 54 teaches “adjusting the coordinates of vertices of the face foam mesh to coordinates of the face scan mesh vertices), and extending mesh edges as needed during this process… fit criteria may additionally or alternatively include outer and/or inner boundary criteria, e.g., requirement that the outer and inner boundaries 1206 and 1204 of the foam mesh can be fitted to the contour of the face data mesh without excessive (e.g., more than 5%) distortion at the outer boundary”); extending mesh edge and having fit criteria shows generating shape (by extending mesh edge) but also keeping within a protrusion percentage/criteria (thus based on protruding point(s)). Lindauer is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of protruding in a mesh and analysis thereof. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Reichow, Russell, Eck and Grundhofer with the protruding mesh techniques of Lindauer to ensure reducing the size of the dataset to be manipulated and thus improving computational efficiency (Lindauer, paragraph 48). This would be due to the protruding/distortion from the percentage criteria and removal of certain areas thereof.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Stewart et al. (U.S. Patent Application Publication No. 2015/0208019) paragraph 16 teaches “orientation of a projector may change with respect to a projection surface and, for example, orientation of a viewer may change with respect to a projection surface. Further, a projection surface may be multifaceted, non-planar, dynamic, etc.”; this shows projection surface dynamic and orientation of projector changes.
Jang et al. (US 20180213192), paragraph 22 teaches “the position of a projector is changed due to projection of the contents image”.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NAUMAN U AHMAD whose telephone number is (703)756-5306. The examiner can normally be reached Monday - Friday 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
/N.U.A./Examiner, Art Unit 2611
Prosecution Insights