DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 0 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant's arguments filed 04/08/2026 have been fully considered but they are not persuasive.
Applicant argues Little et al. (US 12450662 B2) (hereinafter Little) does not disclose
receiving and storing video data
rendering a display video based on overlaying a video depiction of a dynamic action scene involving at least one person of stored second video data with a depiction of a static portion of the environment of first video data
video data
how to render said display video based on determining that a first geographic location of a first device at a first time period is within a proximity threshold of a second geographic location of a second device at a second time period (occurring prior to the first time period)
Examiner replies, Little discloses “The VR/AR/MR rendering device 1204 may spatially capture the actual environment 1206 to generate a data set (video) that is representative of the actual environment 1206” in Col 15 Lines 27-30 Examiner’s note: video is defined as a sequence of digital images; “The VR/AR/MR processor 102 or an analogous device may receive an image of the environment from the VR/AR/MR rendering device 304” in Col 9 Lines 6-8; “Inventory and Claim Database (storing video data) in FIG.1; See FIG.3 and FIG. 12, “FIG. 3 illustrates an example environment 300 as displayed to a user 302 wearing a head-mounted VR/AR/MR rendering device 304.” in Col 8 Lines 8-10”; “FIG. 12 illustrates a view of an actual environment 1206, after an incident, as displayed to a user 1202 on a display of an VR/AR/MR rendering device 1204.” in Col Lines 21-23; Examiner’s note: two figures show the same environment that corresponds the first geographic location is same to the second geographic location.
Regarding the remaining arguments: Applicant argue with respect to the amended claim language, which is fully addresses in the prior art rejection set forth below.
Conclusion: The rejection set in the previous Office Action are shown to have been proper, and the claims are rejected below. New citations and parenthetical remarks
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4, 10-12, 14, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Little et al. (US 12450662 B1) (hereinafter Little) in view of Kurz et al. (US 20250104346 A1) (hereinafter Kurz).
Regarding claim 1, Little disclose a method comprising: (Background, “It is desirable to provide systems and methods to assist insurance policy holders to create and maintain an inventory of property.” identifying, using control circuitry, a first geographic location of a first device during a first time period; (Col 8 Line 2-7, “Certain computing devices may additionally or alternatively include one or more hardware components (e.g., application specific integrated circuits, field programmable gate arrays, systems on a chip, and the like) to implement some or all of the functionalities they are described as performing.”; Col 1 Line 65 - Col 2 Line 10, “a system, comprising one or more processors and memory coupled to the one or more processors. The memory stores instructions executable by the one or more processors to perform operations. The operations include receiving first three-dimensional image data at a first time, the first three-dimensional image data representing an environment and a first plurality of objects disposed within the environment at the first time. The operations further include receiving second three-dimensional image data at a second time, the second time being later than the first time, and the second three-dimensional image data representing the environment at the second time.”)
receiving, using the control circuitry, first video data of an environment captured by the first device at the first geographic location during the first time period, wherein the first video data comprises a depiction of a static portion of the environment of the first geographic location in three dimensions during the first time period; (FIG 12; Col 1 Line 65 - Col 2 Line 10, “a system, comprising one or more processors and memory coupled to the one or more processors. The memory stores instructions executable by the one or more processors to perform operations. The operations include receiving first three-dimensional image data at a first time, the first three-dimensional image data representing an environment and a first plurality of objects disposed within the environment at the first time. The operations further include receiving second three-dimensional image data at a second time, the second time being later than the first time, and the second three-dimensional image data representing the environment at the second time.”; Col 15 Lines 21-43, “FIG. 12 illustrates a view of an actual environment 1206, after an incident, as displayed to a user 1202 on a display of an VR/AR/MR rendering device 1204. The user 1202 and the VR/AR/MR rendering device 1204 are illustrated to show the perspective of the user 1202, but they are depicted in dashed lines to indicate that they are not part of what is displayed to the user 1202. The VR/AR/MR rendering device 1204 may spatially capture the actual environment 1206 to generate a data set that is representative of the actual environment 1206. For example, the VR/AR/MR rendering device 1204 may include an imaging device such as a three-dimensional scanner, and the VR/AR/MR rendering device 1204 may generate a point cloud or other three-dimensional representation that is representative of the actual environment 1206. In FIG. 12, in addition to displaying an image of the actual environment 1206 on a display of the VR/AR/MR rendering device 1204, the VR/AR/MR rendering device 1204 also displays a virtual assistant 1208 that is not present in the actual environment 1206. The virtual assistant 1208 may assist the user 1202 to generate a listing of objects in the actual environment, such as a listing of objects for an insurance claim.” Col 15 Lines 53-65, “The VR/AR/MR processor 102 may cause the VR/AR/MR rendering device 1204 to display the virtual assistant 1208 in the image as pointing to or otherwise non-verbally indicating the window 1210. The VR/AR/MR processor 102 may cause the window 1210 in the image displayed to the user 1202 in a highlighted manner or otherwise visually indicated. This may assist the user 1202 to know that the virtual assistant 1208 is pointing to or otherwise non-verbally indicating the window 1210. The virtual assistant 1208 may, in some examples, also or instead make an utterance 1212, such as inquiring as to whether the window 1210 is damaged. The virtual assistant 1208 may indicate other objects in the environment 1206.”; Col 5 Lines 53-55, “As another example, the objects may include objects that are permanent fixtures in the actual environment, such as doors or windows.” Examiner’s note: window, cabinet, painting, door, walls, and ceiling in FIG. 12 corresponds to the depiction of a static portions.)
identifying, using the control circuitry, a second geographic location of a second device during a second time period, wherein the second time period occurred earlier than the first time period; (Col 1 Line 65 - Col 2 Line 10, “a system, comprising one or more processors and memory coupled to the one or more processors. The memory stores instructions executable by the one or more processors to perform operations. The operations include receiving first three-dimensional image data at a first time, the first three-dimensional image data representing an environment and a first plurality of objects disposed within the environment at the first time. The operations further include receiving second three-dimensional image data at a second time, the second time being later than the first time, and the second three-dimensional image data representing the environment at the second time.”)
based at least in part on determining that the first geographic location is within a proximity threshold of the second geographic location: (FIG3; FIG 12; Col 8 Lines 8-14, “FIG. 3 illustrates an example environment 300 as displayed to a user 302 wearing a head-mounted VR/AR/MR rendering device 304. One or more images of the environment 300 that the head-mounted VR/AR/MR rendering device 304 displays to the user may assist the use to generate an inventory of objects and/or prepare a claim for damage to the objects.”; Col8, Lines 27-32 ,” The head-mounted VR/AR/MR rendering device 304 displays to the user 302 the actual environment 300, such as a living room, and/or a representation of the actual environment 300. In the example shown in FIG. 3, the actual environment 300 includes a cabinet 306, a painting 308 and a window 310.”; Col 15, Lines 21-23, “FIG. 12 illustrates a view of an actual environment 1206, after an incident, as displayed to a user 1202 on a display of an VR/AR/MR rendering device 1204.” Examiner’s note: Two figures show the same environment that corresponds the geographic location is same to the second geographic location.)
accessing, using the control circuitry, stored second video data of the environment captured by the second device during the second time period, wherein the stored second video data comprises: (Col 1 Line 65 - Col 2 Line 10, “a system, comprising one or more processors and memory coupled to the one or more processors. The memory stores instructions executable by the one or more processors to perform operations. The operations include receiving first three-dimensional image data at a first time, the first three-dimensional image data representing an environment and a first plurality of objects disposed within the environment at the first time. The operations further include receiving second three-dimensional image data at a second time, the second time being later than the first time, and the second three-dimensional image data representing the environment at the second time.”; Col5 Lines 43-55, “The VR/AR/MR processor 102 in the example system 100 is connected to an inventory and claim database 124 via a network 126. The inventory and claim database 124 may include, for example, a record of image and audio data received from the output portion 106 of the VR/AR/MR rendering device 104, as well as a repository of information such as metadata about objects in the user's actual environment and/or other environments. The objects may be, for example, objects possessed by the user and which are insured under a property insurance policy against damage, theft and/or other incidents. As another example, the objects may include objects that are permanent fixtures in the actual environment, such as doors or windows.”)
(a) a depiction of a static portion of the environment of the second geographic location in three dimensions, and (Col 1 Line 65 - Col 2 Line 10, “a system, comprising one or more processors and memory coupled to the one or more processors. The memory stores instructions executable by the one or more processors to perform operations. The operations include receiving first three-dimensional image data at a first time, the first three-dimensional image data representing an environment and a first plurality of objects disposed within the environment at the first time. The operations further include receiving second three-dimensional image data at a second time, the second time being later than the first time, and the second three-dimensional image data representing the environment at the second time.”; FIG. 3; ; Col 8 Lines 8-14, “FIG. 3 illustrates an example environment 300 as displayed to a user 302 wearing a head-mounted VR/AR/MR rendering device 304. One or more images of the environment 300 that the head-mounted VR/AR/MR rendering device 304 displays to the user may assist the use to generate an inventory of objects and/or prepare a claim for damage to the objects.”; Col8, Lines 27-32 ,” The head-mounted VR/AR/MR rendering device 304 displays to the user 302 the actual environment 300, such as a living room, and/or a representation of the actual environment 300. In the example shown in FIG. 3, the actual environment 300 includes a cabinet 306, a painting 308 and a window 310.”)
causing, using the control circuitry, a display video to be rendered, based on: (FIFG 12; FIG. 13; FIG. 14; Col 8 Line 18-20, “the depiction of the user 302 and VR/AR/MR rendering device 304 is provided to show the perspective of the view of the environment 300.”; Col 1 Line 53-59, “The techniques further include causing an electronic device to display a three-dimensional image of the environment based at least in part on at least one of the first three-dimensional image data and the second three-dimensional image data, the three-dimensional image including a three-dimensional rendering of the object.”; Col 13 Line 41-46, “The VR/AR/MR rendering device 1004 may be capturing an image of an actual environment in real time, such as obtaining point cloud or other data from which a three-dimensional image representative of the actual environment 1000 may be displayed.”; Col 15 Lines 21-35, “FIG. 12 illustrates a view of an actual environment 1206, after an incident, as displayed to a user 1202 on a display of an VR/AR/MR rendering device 1204. The user 1202 and the VR/AR/MR rendering device 1204 are illustrated to show the perspective of the user 1202, but they are depicted in dashed lines to indicate that they are not part of what is displayed to the user 1202. The VR/AR/MR rendering device 1204 may spatially capture the actual environment 1206 to generate a data set that is representative of the actual environment 1206. For example, the VR/AR/MR rendering device 1204 may include an imaging device such as a three-dimensional scanner, and the VR/AR/MR rendering device 1204 may generate a point cloud or other three-dimensional representation that is representative of the actual environment 1206.”)
Little does not explicitly disclose a video depiction of a dynamic action scene involving at least one person during the second time period and overlaying the video depiction of the dynamic action scene involving at least one person during the second time period.
However, Kurz further teaches video data (para. [0059], “video (e.g., pass-through video depicting a physical environment) is received from an image sensor of a device (e.g., device 405) and used to present the XR environment.)
Kurz more explicitly teaches a video depiction of a dynamic action scene involving at least one person during the second time (para. [0044], “the one or more sensors 116 may capture images of the user's forehead, eyebrows, eyes, eye lids, cheeks, nose, lips, chin, face, head, hands, wrists, arms, shoulders, torso, legs, or other body portion”)
overlaying the video depiction of the dynamic action scene involving at least one person during the second time period with the depiction of the static portion of the environment of the first geographic location (room) in three dimensions during the first time period with the depiction of the static portion of the environment of the first geographic location (room) in three dimensions during the first time period. (FIG.4, FIG. 5A, FIG. 5B, para. [0059], “In other implementations, optical see-through may be used to present the XR environment by overlaying virtual content on a view of the physical environment seen through a translucent or transparent display. In some implementations, a 3D representation of a virtual environment is aligned with a 3D coordinate system of the physical environment.”)
As both Little and Kurz are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include a video depiction of a dynamic action scene involving at least one person during the second time period and overlaying the video depiction of the dynamic action scene involving at least one person during the second time period in the context of image processing, by Little according to the teaching of Kurz in order to conserve power and computing resources used to generate a temporal-based three-dimensional (3D) representation for use in providing/rendering views of the recorded event (Background of Kurz).
Regarding claim 2, the combination of Little and Kurz disclose the method of claim 1, wherein the video depiction of the dynamic action scene involving at least one person during the second time period is disposed at a position in the environment of the stored second video data, (see supra rejection of claim 1)
Little discloses wherein the rendering comprises rendering the video (Col 1 Line 51-59, “The techniques further include identifying an object of the first plurality of objects based at least in part on the difference. The techniques further include causing an electronic device to display a three-dimensional image of the environment based at least in part on at least one of the first three-dimensional image data and the second three-dimensional image data, the three-dimensional image including a three-dimensional rendering of the object.”)
Little does not disclose the video depiction of the dynamic action scene involving at least one person
However, Kurz more explicitly teaches wherein the rendering comprises rendering the video depiction of the dynamic action scene involving at least one person (para. [0044], “the one or more sensors 116 may capture images of the user's forehead, eyebrows, eyes, eye lids, cheeks, nose, lips, chin, face, head, hands, wrists, arms, shoulders, torso, legs, or other body portion”;
As both Little and Kurz are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the video depiction of the dynamic action scene involving at least one person, in the context of image processing, by Little according to the teaching of Kurz in order to conserve power and computing resources used to generate a temporal-based three-dimensional (3D) representation for use in providing/rendering views of the recorded event (Background of Kurz).
Regarding claim 4, the combination of Little and Kurz disclose the method of claim 1, wherein accessing the stored second video data comprises: (see supra rejection of claim 1)
Little does not explicitly disclose selecting the stored second video data from a plurality of stored second video data based on determining that the environment captured by the first video data is depicted in the same orientation as the environment characterized captured by the second video data.
Kurz more explicitly teaches selecting the stored second video data from a plurality of stored second video data based on determining that the environment captured by the first video data is depicted in the same orientation as the environment characterized captured by the second video data. (Image Content Database(s) 330 in FIG. 3; para. [0053], “the content instruction set 306 of the information system 104 collects the recorded content 335 from the one or more devices and stores the content in the one or more image content database(s) 330. For instance, information system 104 can generate a complete 3D representation of the event based on 2D and/or 3D images captured by devices by rendering views corresponding to multiple instances of time during a time period of a recorded event (e.g., 60 seconds of different views may be provided that correspond to a 60 second recorded event, with a first view provided for a first time point during the event, a second view provided for the next time point during the event, etc.)”; para. [0113], “At block 910, the method 900 determines a 3D event viewpoint corresponding to a 3D position in a physical environment of a recorded event. For example, as illustrated in FIG. 7, a neural rendering model may include continuous scene construction that obtains as an input a 5D vector-valued function with the following characteristics for the input: a 3D location x=(x; y; z) and a 2D viewing direction (θ; ϕ). In some implementations, the event viewpoint may be specified by the renderer; the viewpoint may be based on rendering device pose and thus may change for different rendering times based on current rendering device position and/or orientation (e.g., current HMD pose); FIG 7; FIG 9)
As both Little and Kurz are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include selecting the stored second video data from a plurality of stored second video data based on determining that the environment captured by the first video data is depicted in the same orientation as the environment characterized captured by the second video data, in the context of image processing, by Little according to the teaching of Kurz in order to conserve power and computing resources used to generate a temporal-based three-dimensional (3D) representation for use in providing/rendering views of the recorded event (Background of Kurz).
Regarding claim 10, the combination of Little and Kurz disclose the method of claim 1, wherein the accessing the stored second video data of the environment is performed based at least in part on detecting co-location of the first device with a virtual object in the environment. (Little, Col 5 Lines 43-55, “The VR/AR/MR processor 102 in the example system 100 is connected to an inventory and claim database 124 via a network 126. The inventory and claim database 124 may include, for example, a record of image and audio data received from the output portion 106 of the VR/AR/MR rendering device 104, as well as a repository of information such as metadata about objects in the user's actual environment and/or other environments. The objects may be, for example, objects possessed by the user and which are insured under a property insurance policy against damage, theft and/or other incidents. As another example, the objects may include objects that are permanent fixtures in the actual environment, such as doors or windows.”; FIG3; FIG 12; Col 8 Lines 8-14, “FIG. 3 illustrates an example environment 300 as displayed to a user 302 wearing a head-mounted VR/AR/MR rendering device 304. One or more images of the environment 300 that the head-mounted VR/AR/MR rendering device 304 displays to the user may assist the use to generate an inventory of objects and/or prepare a claim for damage to the objects.”; Col8, Lines 27-32 ,” The head-mounted VR/AR/MR rendering device 304 displays to the user 302 the actual environment 300, such as a living room, and/or a representation of the actual environment 300. In the example shown in FIG. 3, the actual environment 300 includes a cabinet 306, a painting 308 and a window 310.”; Col 15, Lines 21-23, “FIG. 12 illustrates a view of an actual environment 1206, after an incident, as displayed to a user 1202 on a display of an VR/AR/MR rendering device 1204.” Examiner’s note: Two figures show the same environment that corresponds the geographic location is same to the second geographic location.; FIG 13; Col 15 Lines 53-65, “The VR/AR/MR processor 102 may cause the VR/AR/MR rendering device 1204 to display the virtual assistant 1208 in the image as pointing to or otherwise non-verbally indicating the window 1210. The VR/AR/MR processor 102 may cause the window 1210 in the image displayed to the user 1202 in a highlighted manner or otherwise visually indicated.”; Col 15 Line 66 -Col 16 Line 17 “FIG. 13 illustrates another example of a view of the actual environment 1206 shown in FIG. 12 that the VR/AR/MR rendering device 1204 displays to the user 1202. The view the VR/AR/MR rendering device 1204 displays to the user in FIG. 13 is similar to the view the VR/AR/MR rendering device 1204 displays to the user in FIG. 12. In the example shown in FIG. 13, while the VR/AR/MR rendering device 1204 displays the view, the VR/AR/MR rendering device 1204 displays an indication 1302 that a painting was determined to be missing, including a request 1304 for the user to confirm. The user 1202 also makes an utterance 1306 indicating that the window 1210 is damaged and confirming that a painting is missing. The user 1202 also states in the utterance 1306, based on observation by the user 1202 and not automatically detected, that a drawer is missing in the cabinet 1308. Displaying the indications and accepting input from the user may assist in the generation of an accurate and complete listing of damaged objects in the environment 1206.”)
Regarding claim 11, Little discloses a system comprising: (Col 1 Line 65, “This disclosure describes, in another aspect, a system..”)
a first device comprising a display arranged to be viewed by a viewer; (FIG. 14)
PNG
media_image1.png
529
676
media_image1.png
Greyscale
a second device (FIG. 9); and
PNG
media_image2.png
548
674
media_image2.png
Greyscale
control circuitry configured to: (Col 1 Line 65-66, “This disclosure describes, in another aspect, a system, comprising one or more processors”)
identify, a first geographic location of a first device during a first time period; (Col 8 Line 2-7, “Certain computing devices may additionally or alternatively include one or more hardware components (e.g., application specific integrated circuits, field programmable gate arrays, systems on a chip, and the like) to implement some or all of the functionalities they are described as performing.”; Col 1 Line 65 - Col 2 Line 10, “a system, comprising one or more processors and memory coupled to the one or more processors. The memory stores instructions executable by the one or more processors to perform operations. The operations include receiving first three-dimensional image data at a first time, the first three-dimensional image data representing an environment and a first plurality of objects disposed within the environment at the first time. The operations further include receiving second three-dimensional image data at a second time, the second time being later than the first time, and the second three-dimensional image data representing the environment at the second time.”)
receive first video data of an environment captured by the first device at the first geographic location during the first time period, wherein the first video data comprises a depiction of a static portion of the environment of the first geographic location in three dimensions during the first time period; (FIG 12; Col 1 Line 65 - Col 2 Line 10, “a system, comprising one or more processors and memory coupled to the one or more processors. The memory stores instructions executable by the one or more processors to perform operations. The operations include receiving first three-dimensional image data at a first time, the first three-dimensional image data representing an environment and a first plurality of objects disposed within the environment at the first time. The operations further include receiving second three-dimensional image data at a second time, the second time being later than the first time, and the second three-dimensional image data representing the environment at the second time.”; Col 15 Lines 21-43, “FIG. 12 illustrates a view of an actual environment 1206, after an incident, as displayed to a user 1202 on a display of an VR/AR/MR rendering device 1204. The user 1202 and the VR/AR/MR rendering device 1204 are illustrated to show the perspective of the user 1202, but they are depicted in dashed lines to indicate that they are not part of what is displayed to the user 1202. The VR/AR/MR rendering device 1204 may spatially capture the actual environment 1206 to generate a data set that is representative of the actual environment 1206. For example, the VR/AR/MR rendering device 1204 may include an imaging device such as a three-dimensional scanner, and the VR/AR/MR rendering device 1204 may generate a point cloud or other three-dimensional representation that is representative of the actual environment 1206. In FIG. 12, in addition to displaying an image of the actual environment 1206 on a display of the VR/AR/MR rendering device 1204, the VR/AR/MR rendering device 1204 also displays a virtual assistant 1208 that is not present in the actual environment 1206. The virtual assistant 1208 may assist the user 1202 to generate a listing of objects in the actual environment, such as a listing of objects for an insurance claim.” Col 15 Lines 53-65, “The VR/AR/MR processor 102 may cause the VR/AR/MR rendering device 1204 to display the virtual assistant 1208 in the image as pointing to or otherwise non-verbally indicating the window 1210. The VR/AR/MR processor 102 may cause the window 1210 in the image displayed to the user 1202 in a highlighted manner or otherwise visually indicated. This may assist the user 1202 to know that the virtual assistant 1208 is pointing to or otherwise non-verbally indicating the window 1210. The virtual assistant 1208 may, in some examples, also or instead make an utterance 1212, such as inquiring as to whether the window 1210 is damaged. The virtual assistant 1208 may indicate other objects in the environment 1206.”; Col 5 Lines 53-55, “As another example, the objects may include objects that are permanent fixtures in the actual environment, such as doors or windows.” Examiner’s note: window, cabinet, painting, door, walls, and ceiling in FIG. 12 corresponds to the depiction of a static portions.)
identify a second geographic location of a second device during a second time period, wherein the second time period occurred earlier than the first time period; (Col 1 Line 65 - Col 2 Line 10, “a system, comprising one or more processors and memory coupled to the one or more processors. The memory stores instructions executable by the one or more processors to perform operations. The operations include receiving first three-dimensional image data at a first time, the first three-dimensional image data representing an environment and a first plurality of objects disposed within the environment at the first time. The operations further include receiving second three-dimensional image data at a second time, the second time being later than the first time, and the second three-dimensional image data representing the environment at the second time.”)
based at least in part on determining that the first geographic location is within a proximity threshold of the second geographic location: (FIG3; FIG 12; Col 8 Lines 8-14, “FIG. 3 illustrates an example environment 300 as displayed to a user 302 wearing a head-mounted VR/AR/MR rendering device 304. One or more images of the environment 300 that the head-mounted VR/AR/MR rendering device 304 displays to the user may assist the use to generate an inventory of objects and/or prepare a claim for damage to the objects.”; Col8, Lines 27-32 ,” The head-mounted VR/AR/MR rendering device 304 displays to the user 302 the actual environment 300, such as a living room, and/or a representation of the actual environment 300. In the example shown in FIG. 3, the actual environment 300 includes a cabinet 306, a painting 308 and a window 310.”; Col 15, Lines 21-23, “FIG. 12 illustrates a view of an actual environment 1206, after an incident, as displayed to a user 1202 on a display of an VR/AR/MR rendering device 1204.” Examiner’s note: Two figures show the same environment that corresponds the geographic location is same to the second geographic location.)
access stored second video data of the environment captured by the second device during the second time period, wherein the stored second video data comprises: (Col 1 Line 65 - Col 2 Line 10, “a system, comprising one or more processors and memory coupled to the one or more processors. The memory stores instructions executable by the one or more processors to perform operations. The operations include receiving first three-dimensional image data at a first time, the first three-dimensional image data representing an environment and a first plurality of objects disposed within the environment at the first time. The operations further include receiving second three-dimensional image data at a second time, the second time being later than the first time, and the second three-dimensional image data representing the environment at the second time.”)
(a) a depiction of a static portion of the environment of the second geographic location in three dimensions, and (Col 1 Line 65 - Col 2 Line 10, “a system, comprising one or more processors and memory coupled to the one or more processors. The memory stores instructions executable by the one or more processors to perform operations. The operations include receiving first three-dimensional image data at a first time, the first three-dimensional image data representing an environment and a first plurality of objects disposed within the environment at the first time. The operations further include receiving second three-dimensional image data at a second time, the second time being later than the first time, and the second three-dimensional image data representing the environment at the second time.”; FIG. 3; ; Col 8 Lines 8-14, “FIG. 3 illustrates an example environment 300 as displayed to a user 302 wearing a head-mounted VR/AR/MR rendering device 304. One or more images of the environment 300 that the head-mounted VR/AR/MR rendering device 304 displays to the user may assist the use to generate an inventory of objects and/or prepare a claim for damage to the objects.”; Col8, Lines 27-32 ,” The head-mounted VR/AR/MR rendering device 304 displays to the user 302 the actual environment 300, such as a living room, and/or a representation of the actual environment 300. In the example shown in FIG. 3, the actual environment 300 includes a cabinet 306, a painting 308 and a window 310.”)
render, a display video on the display of the first device based on:
(FIFG 12; FIG. 13; FIG. 14; Col 8 Line 18-20, “the depiction of the user 302 and VR/AR/MR rendering device 304 is provided to show the perspective of the view of the environment 300.”; Col 1 Line 53-59, “The techniques further include causing an electronic device to display a three-dimensional image of the environment based at least in part on at least one of the first three-dimensional image data and the second three-dimensional image data, the three-dimensional image including a three-dimensional rendering of the object.”; Col 13 Line 41-46, “The VR/AR/MR rendering device 1004 may be capturing an image of an actual environment in real time, such as obtaining point cloud or other data from which a three-dimensional image representative of the actual environment 1000 may be displayed.”; Col 15 Lines 21-35, “FIG. 12 illustrates a view of an actual environment 1206, after an incident, as displayed to a user 1202 on a display of an VR/AR/MR rendering device 1204. The user 1202 and the VR/AR/MR rendering device 1204 are illustrated to show the perspective of the user 1202, but they are depicted in dashed lines to indicate that they are not part of what is displayed to the user 1202. The VR/AR/MR rendering device 1204 may spatially capture the actual environment 1206 to generate a data set that is representative of the actual environment 1206. For example, the VR/AR/MR rendering device 1204 may include an imaging device such as a three-dimensional scanner, and the VR/AR/MR rendering device 1204 may generate a point cloud or other three-dimensional representation that is representative of the actual environment 1206.”)
Little does not explicitly disclose a video depiction of a dynamic action scene involving at least one person during the second time period and overlaying the video depiction of the dynamic action scene involving at least one person during the second time period.
However, Kurz further teaches video data (para. [0059], “video (e.g., pass-through video depicting a physical environment) is received from an image sensor of a device (e.g., device 405) and used to present the XR environment.)
Kurz more explicitly teaches a video depiction of a dynamic action scene involving at least one person during the second time (para. [0044], “the one or more sensors 116 may capture images of the user's forehead, eyebrows, eyes, eye lids, cheeks, nose, lips, chin, face, head, hands, wrists, arms, shoulders, torso, legs, or other body portion”)
overlaying the video depiction of the dynamic action scene involving at least one person during the second time period with the depiction of the static portion of the environment of the first geographic location (room) in three dimensions during the first time period with the depiction of the static portion of the environment of the first geographic location (room) in three dimensions during the first time period. (FIG.4, FIG. 5A, FIG. 5B, para. [0059], “In other implementations, optical see-through may be used to present the XR environment by overlaying virtual content on a view of the physical environment seen through a translucent or transparent display. In some implementations, a 3D representation of a virtual environment is aligned with a 3D coordinate system of the physical environment.”)
As both Little and Kurz are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include a video depiction of a dynamic action scene involving at least one person during the second time period and overlaying the video depiction of the dynamic action scene involving at least one person during the second time period in the context of image processing, by Little according to the teaching of Kurz in order to conserve power and computing resources used to generate a temporal-based three-dimensional (3D) representation for use in providing/rendering views of the recorded event (Background of Kurz).
Regarding claim 12, claim 12 is system claim of claim 2 except control circuitry (Col 1 Line 65-66, “This disclosure describes, in another aspect, a system, comprising one or more processors”) and is accordingly rejected under same rationale.
Regarding claim 14, claim 14 is system claim of claim 4 except control circuitry (Col 1 Line 65-66, “This disclosure describes, in another aspect, a system, comprising one or more processors”) and is accordingly rejected under same rationale.
Regarding claim 20, the combination of Little and Kurz discloses the system of claim 11, where in the first device is an extended reality device, and where the control circuitry configured to:
access the stored second video data of the environment based at least in part on detecting co-location of the extended reality device with a virtual object in the environment. (Col 1 Line 65-66, “This disclosure describes, in another aspect, a system, comprising one or more processors”; Col 5 Lines 43-55, “The VR/AR/MR processor 102 in the example system 100 is connected to an inventory and claim database 124 via a network 126. The inventory and claim database 124 may include, for example, a record of image and audio data received from the output portion 106 of the VR/AR/MR rendering device 104, as well as a repository of information such as metadata about objects in the user's actual environment and/or other environments. The objects may be, for example, objects possessed by the user and which are insured under a property insurance policy against damage, theft and/or other incidents. As another example, the objects may include objects that are permanent fixtures in the actual environment, such as doors or windows.”; FIG3; FIG 12; Col 8 Lines 8-14, “FIG. 3 illustrates an example environment 300 as displayed to a user 302 wearing a head-mounted VR/AR/MR rendering device 304. One or more images of the environment 300 that the head-mounted VR/AR/MR rendering device 304 displays to the user may assist the use to generate an inventory of objects and/or prepare a claim for damage to the objects.”; Col8, Lines 27-32 ,” The head-mounted VR/AR/MR rendering device 304 displays to the user 302 the actual environment 300, such as a living room, and/or a representation of the actual environment 300. In the example shown in FIG. 3, the actual environment 300 includes a cabinet 306, a painting 308 and a window 310.”; Col 15, Lines 21-23, “FIG. 12 illustrates a view of an actual environment 1206, after an incident, as displayed to a user 1202 on a display of an VR/AR/MR rendering device 1204.” Examiner’s note: Two figures show the same environment that corresponds the geographic location is same to the second geographic location.; FIG 13; Col 15 Lines 53-65, “The VR/AR/MR processor 102 may cause the VR/AR/MR rendering device 1204 to display the virtual assistant 1208 in the image as pointing to or otherwise non-verbally indicating the window 1210. The VR/AR/MR processor 102 may cause the window 1210 in the image displayed to the user 1202 in a highlighted manner or otherwise visually indicated.”; Col 15 Line 66 -Col 16 Line 17 “FIG. 13 illustrates another example of a view of the actual environment 1206 shown in FIG. 12 that the VR/AR/MR rendering device 1204 displays to the user 1202. The view the VR/AR/MR rendering device 1204 displays to the user in FIG. 13 is similar to the view the VR/AR/MR rendering device 1204 displays to the user in FIG. 12. In the example shown in FIG. 13, while the VR/AR/MR rendering device 1204 displays the view, the VR/AR/MR rendering device 1204 displays an indication 1302 that a painting was determined to be missing, including a request 1304 for the user to confirm. The user 1202 also makes an utterance 1306 indicating that the window 1210 is damaged and confirming that a painting is missing. The user 1202 also states in the utterance 1306, based on observation by the user 1202 and not automatically detected, that a drawer is missing in the cabinet 1308. Displaying the indications and accepting input from the user may assist in the generation of an accurate and complete listing of damaged objects in the environment 1206.”)
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Little et al. (US 12450662 B1) (hereinafter Little) in view of Kurz et al. (US 20250104346 A1) (hereinafter Kurz), and further in view of Chen et al. (US 20240144620 A1) (Hereinafter Chen).
Regarding claim 8, the combination of Little and Kurz disclose the method of claim 1, further comprising:
modifying the video depiction of the dynamic action scene involving at least one person of the stored second video data (Kurz, para. [0044], “the one or more sensors 116 may capture images of the user's forehead, eyebrows, eyes, eye lids, cheeks, nose, lips, chin, face, head, hands, wrists, arms, shoulders, torso, legs, or other body portion”)
wherein the display video comprises the modified video depiction of the dynamic action scene involving at least one person of the stored second video data. (Kurz, FIG. 7; para. [0083], “FIG. 7 illustrates an example environment 700 for implementing a process for generating a temporal-based 3D representation of a shared event from a volume rendering model based on expiration values, in accordance with some implementations. The example environment 700 may include a viewing device 405, a temporal-based 3D representation instruction set 624, and a volume rendering model instruction set 622 that communicates over a data communication network (e.g., network 602), e.g., a LAN, a WAN, the Internet, a mobile network, or a combination thereof.” para. [0089], “Then the output data 765 may be used by the temporal-based 3D representation instruction set 62 to generate a temporal-based 3D representation 770 (e.g., a rendering of the party as illustrated in FIG. 1) that the user 402 can view different viewpoints in the dynamic scene that were synthesized from different content from different viewpoints.”; para. [0044], “the one or more sensors 116 may capture images of the user's forehead, eyebrows, eyes, eye lids, cheeks, nose, lips, chin, face, head, hands, wrists, arms, shoulders, torso, legs, or other body portion”)
The combination of Little and Kurz does not disclose determining one or more environmental context values from the first video data and based on the one or more environmental context values.
Chen more explicitly discloses determining one or more environmental context values from the first video data and based on the one or more environmental context values. (para. [0087], “The input images may be the output representative data (accessed video essence and metadata) of the moment in VR. Additional visual data may provide texture, color, and context information to reduce inaccuracy and inconsistency in depth estimation and inpainting. Improved depth values may help in rendering new views.”; para. [0091], “In some embodiments, having additional texture, color, context and depth information from capturing the VR moment may improve depth estimation and may improve synthesizing new views to complete the point cloud.”)
As Little, Kurz, and Chen from the same filed of endeavor, it would have been obvious to one of ordinary skill in the art before the effective fling date of the claimed invention to include determining one or more environmental context values from the first video data and based on the one or more environmental context values, in context of image processing, disclosed by Little and Kurz according to the teachings of Chen in order to enable creation of an enhanced image (e.g., 2D or 3D images, photos, videos) of a view of a VR environment (para. [0005] of Chen).
Regarding claim 18, claim 18 is the system claim of claim 8 and is accordingly rejected under same rationale.
Claims 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Little et al. (US 12450662 B1) (hereinafter Little) in view of Kurz et al. (US 20250104346 A1) (hereinafter Kurz), and further in view of Cansizoglu et al. (US 12307605 B2) (Hereinafter Cansizoglu).
Regarding claim 9, the combination of Little and Kurz disclose the method of claim 1, further comprising: (Kurz, para. [0044], “the one or more sensors 116 may capture images of the user's forehead, eyebrows, eyes, eye lids, cheeks, nose, lips, chin, face, head, hands, wrists, arms, shoulders, torso, legs, or other body portion”, para. [0053], “In an exemplary implementation, the content instruction set 306 of the information system 104 collects the recorded content 335 from the one or more devices and stores the content in the one or more image content database(s) 330.”)
The combination of Little and Kurz does not disclose generating third video data comprising the first video data.
However, Cansizoglu more explicitly teaches generating third video data comprising the first video data. (Col 31 Line 35-37, “third image data based on the first metadata and the second image processing operation performed on the second image data.”)
As both Little, Kurz, and Cansizoglu from the same filed of endeavor, it would have been obvious to one of ordinary skill in the art before the effective fling date of the claimed invention to include generating third video data comprising the first video data, in context of image processing, disclosed by Little and Kurz according to the teachings of Cansizoglu in order to enable real and virtual environments to be combined in varying degrees to facilitate interactions from a user in a real time manner (see Col 2 Line 36-38 of Cansizoglu).
Regarding claim 19, claim 19 is the system claim of claim 9 and is accordingly rejected under same rationale.
Allowable Subject Matter
Claim 5-7, 15-17, and 51-52 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the allowance:
Regarding claim 5, the cited prior art in combination or alone fail to teach the method of claim 1, further comprising: extracting a first plurality of features from the first video data during the first time period, wherein the first plurality of features comprises the depiction of the static portion of the environment of the first geographic location in three dimensions; extracting a second plurality of features from the stored second video data during the second time period, wherein the second plurality of features comprises:
(a) the depiction of the static portion of the environment of the second geographic location in three dimensions, and (b) the video depiction of the dynamic action scene involving at least one person; storing the second plurality of features with the stored second video data; and comparing the first plurality of features with the stored second plurality of features: (a) the depiction of the static portion of the environment of the second geographic location in three dimensions as one or more matched features of the first video data and the stored second video data, and (b) the video depiction of the dynamic action scene involving at least one person as one or more unmatched features of the stored second video data.
Regarding claim 6 and 7, claim 6 and 7 are objected for its dependent from claim 5.
Regarding claim 15, claim 15 is similar in scope to claim 5 and is objected under same rationale as claim 5.
Regarding claim 16 and 17, claim 6 and 7 are objected for its dependent from claim 15.
Regarding claim 51, the cited prior art in combination or alone fail to teach the method of claim 4, wherein accessing the stored second video data comprises: based on (a) determining an orientation change in the environment captured by the first video data and (b) determining that the environment captured by a stored third video data is the same orientation as the changed orientation of the environment captured by the first video data: selecting the stored third video data from the plurality of stored second video data to replace the stored second video for rendering the display video.
Regarding claim 52, claim 52 is the system claim of claim 51 and is accordingly rejected under same rationale.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hyorim Park whose telephone number is (571)272-3859. The examiner can normally be reached Monday - Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Hyorim Park/Examiner, Art Unit 2615
/ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615
Prosecution Insights