DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO internet Web site contains terminal disclaimer forms which may be used. Please visit http://www.uspto.gov/forms/. The filing date of the application will determine what form should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claim 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 and 16 of U.S. Patent No. 12277632. Although the claims at issue are not identical, they are not patentably distinct from each other because claim 1 and 16 in U.S. Patent No. 12277632 defines prior to activating the first and second cameras to capture the first and second videos, determining that the second camera is currently activated and being used to display an image; while the second camera is currently activated and being used to display an image, receiving a request to activate the first and second cameras simultaneously; in response to receiving the request to activate the first and second cameras simultaneously, storing an indication that the second camera was active before the request to activate the first and second cameras simultaneously was received; after generating the AR output, receiving a request to stop simultaneous activation of the first and second cameras; and in response to receiving the request to stop simultaneous activation of the first and second cameras, using the stored indication that the second camera was active before the request to activate the first and second cameras simultaneously was received to deactivate the first camera while maintaining the second camera active. It would have been obvious to one of ordinary skill in the art to notice claim 1 and 18-19 in current application and claim 1 and 16 in U.S. Patent No. 12277632 are almost identical, and claims in current application are much broader. All the claims are essentially the same structure and perform essentially the same function, therefore unpatentable for obvious-type double patenting.
Claim in Application 19073845
Claim in Patent 12277632
1. A method comprising:
1. A method comprising: searching a plurality of augmented reality (AR) experience bundles to identify a subset of AR experience bundles that is associated with simultaneous activation of first and second cameras of a device; displaying icons representing the subset of AR experience bundles, one or more of the displayed icons comprising a camera icon with forward and backward arrows indicative of simultaneous activation of the first and second cameras; launching a first AR experience bundle in response to detecting selection of an icon representing the first AR experience bundle; capturing, by the device, a first video stream using the first camera of the device and a second video stream using the second camera of the device, the first video stream and the second video stream captured simultaneously, the first camera comprising a first image sensor and the second camera comprising a second image sensor, the first image sensor being pointed in an opposite direction from the second image sensor and the first and second image sensors being activated simultaneously; generating an AR output by using the first video stream as a primary input to the first AR experience bundle and using the second video stream as a secondary input to the first AR experience bundle; causing presentation of the AR output on a display of the device; obtaining lighting information from the second image sensor that was used to capture the second video stream; and modifying one or more display attributes of images obtained by the first image sensor based on the lighting information from the second image sensor used to capture the second video stream.
prior to activating first and second cameras to capture first and second videos, determining that the second camera is currently activated and being used to display an image;
16. The method of claim 1, further comprising: prior to activating the first and second cameras to capture the first and second videos, determining that the second camera is currently activated and being used to display an image;
while the second camera is currently activated and being used to display an image, receiving a request to activate the first and second cameras simultaneously;
while the second camera is currently activated and being used to display an image, receiving a request to activate the first and second cameras simultaneously;
in response to receiving the request to activate the first and second cameras simultaneously, storing an indication that the second camera was active before the request to activate the first and second cameras simultaneously was received;
in response to receiving the request to activate the first and second cameras simultaneously, storing an indication that the second camera was active before the request to activate the first and second cameras simultaneously was received;
after generating augmented reality (AR) output, receiving a request to stop simultaneous activation of the first and second cameras; and
after generating the AR output, receiving a request to stop simultaneous activation of the first and second cameras; and
in response to receiving the request to stop simultaneous activation of the first and second cameras, using the stored indication that the second camera was active before the request to activate the first and second cameras simultaneously was received to deactivate the first camera while maintaining the second camera active.
in response to receiving the request to stop simultaneous activation of the first and second cameras, using the stored indication that the second camera was active before the request to activate the first and second cameras simultaneously was received to deactivate the first camera while maintaining the second camera active.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1, 8, 10 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Lee U.S. Patent Application 20190037173 in view of Large U.S. Patent Application 20210144313, and further in view of Luskin U.S. Patent Application 20050030379.
Regarding claim 18, Lee discloses a system comprising:
one or more computer processors (processor 120); and
one or more computer-readable mediums (memory 130) storing instruction that, when executed by the one or more computer processors, cause the system to perform operations (paragraph [0151]: command stored in a computer-readable storage medium in a programming module form. The instruction, when executed by a processor (e.g., the processor 120), may cause the one or more processors to execute the function corresponding to the instruction. The computer-readable storage medium may be, for example, the memory 130) comprising:
prior to activating first and second cameras to capture first and second videos, determining that the second camera is currently activated and being used to display an image (paragraph [0513]: the controller 580 may identify a camera operation state. For example, the controller 580 may identify an on or off state of the camera; paragraph [0761]: when the camera is turned on, the corresponding video may be changed to an image associated with an image which is received from a designated camera);
while the second camera is currently activated and being used to display an image, receiving a request to activate the first and second cameras simultaneously (paragraph [0790]: The user of the first electronic device may select (e.g., touch or tap) a dual camera object 5110 (e.g., dual camera icon) for executing the dual camera, or the first electronic device may execute the narration mode in response to a user's voice input. When the narration mode is executed, the first electronic device may drive the front camera simultaneously in a state in which the rear camera is activated);
after generating augmented reality (AR) output, receiving a request to stop simultaneous activation of the first and second cameras (paragraph [0975]: the first electronic device 6210 may switch the camera driving mode to the camera off mode; paragraph [1046]: it is possible to provide an augmented reality (AR) information providing function, for example, image recognition and related information).
Lee discloses all the features with respect to claim 18 as outlined above. However, Lee fails to disclose in response to receiving the request to activate the first and second cameras simultaneously, storing an indication that the second camera was active before the request to activate the first and second cameras simultaneously was received; in response to receiving the request to stop simultaneous activation of the first and second cameras, using the stored indication that the second camera was active before the request to activate the first and second cameras simultaneously was received to deactivate the first camera while maintaining the second camera active.
Large discloses in response to receiving the request to activate the first and second cameras simultaneously, storing an indication that the second camera was active (paragraph [0087]: computing device 102, 300 or 900 may receive an indication to activate a front and/or rear logo camera, e.g., based on user interaction with an input device for computing device 102, 300 or 900, such as user selection of a logo camera in a touch screen menu; paragraph [0060]: Rear-facing logo camera icon/color filter 206 may provide an in-use notification (indication) (e.g. when logo camera is ON)),
in response to receiving the request to stop simultaneous activation of the first and second cameras, to deactivate the first camera while maintaining the second camera active (paragraph [0094]: In step 818, an indication may be received to turn off (front- and/or rear-facing) camera(s)... receive an indication to deactivate a front and/or rear logo camera, e.g., based on user interaction with an input device for computing device 102, 300 or 900, such as user selection of an icon to close a logo camera in a touch screen menu).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee’s to provide indication as taught by Large, to obtain desired images.
Lee as modified by Large discloses all the features with respect to claim 18 as outlined above. However, Lee as modified by Large fails to disclose storing an indication that the second camera was active before the request to activate the first camera was received; using the stored indication that the second camera was active before the request to activate the first and second cameras simultaneously was received to deactivate the first camera while maintaining the second camera active.
Luskin discloses storing an indication that the second camera was active before the request to activate the first camera was received; using the stored indication that the second camera was active before the request to activate the first and second cameras simultaneously was received to deactivate the first camera while maintaining the second camera active (paragraph [0040]: the default mode can be set to a full-sized rear view and two small windows for left and right cameras. For a system with four cameras, this means that the default mode for left and right cameras are `Window`, the default mode for rear view is `Full` (second camera was active) and the default for front camera is `Off` (deactivate the first camera)).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee and Large’s to set rear camera default mode active as taught by Luskin, to provide video management.
Claim 1 recites the functions of the apparatus recited in claim 18 as method steps. Accordingly, the mapping of the prior art to the corresponding functions of the apparatus in claim 18 applies to the method steps of claim 1.
Regarding claim 8, Lee as modified by Large and Luskin discloses the method of claim 1, wherein the first camera is a front-facing camera on a display side of the device and the second camera is a rear-facing camera on a back side of the device (Lee’s paragraph [0278]: the electronic device 500 may include a front camera (e.g., camera A 810) and a rear camera (e.g., camera B 820) in such a manner that they face each other).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee’s to provide indication as taught by Large, to obtain desired images; and combine Lee and Large’s to set rear camera default mode active as taught by Luskin, to provide video management.
Regarding claim 10, Lee as modified by Large and Luskin discloses the method of claim 1, further comprising:
receiving a request to view a list of augmented reality (AR) experience bundles (Lee’s paragraph [0750]: the view option 4560 may indicate an option to display media data through a thumbnail view, a list view, or an item view and to execute (e.g., full display or the like) the displayed media data; paragraph [1046]: it is possible to provide an augmented reality (AR) information providing function, for example, image recognition and related information); and
while at least one of the first camera and the second camera is not capturing video, receiving an input that selects an option associated with simultaneous activation of the first camera and the second camera (Luskin’s paragraph [0040]: the default mode can be set to a full-sized rear view and two small windows for left and right cameras. For a system with four cameras, this means that the default mode for left and right cameras are `Window`, the default mode for rear view is `Full` and the default for front camera is `Off`; Lee’s paragraph [0790]: The user of the first electronic device may select (e.g., touch or tap) a dual camera object 5110 (e.g., dual camera icon) for executing the dual camera, or the first electronic device may execute the narration mode in response to a user's voice input. When the narration mode is executed, the first electronic device may drive the front camera simultaneously in a state in which the rear camera is activated).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Shirmohamadi and Borremans’s to control camera as taught by Brown, to control capturing of the multiple digital contents using the multiple content capturing devices.
Claim 19 recites the functions of the apparatus recited in claim 18 as medium steps. Accordingly, the mapping of the prior art to the corresponding functions of the apparatus in claim 18 applies to the medium steps of claim 19.
Claim 20 recites the functions of the method recited in claim 8 as medium steps. Accordingly, the mapping of the prior art to the corresponding functions of the method in claim 8 applies to the medium steps of claim 20.
Claim 2-4 are rejected under 35 U.S.C. 103 as being unpatentable over Lee U.S. Patent Application 20190037173 in view of Large U.S. Patent Application 20210144313, in view of Luskin U.S. Patent Application 20050030379, and further in view of Park U.S. Patent Application 20210385417.
Regarding claim 2, Lee as modified by Large and Luskin discloses all the features with respect to claim 1 as outlined above. However, Lee as modified by Large and Luskin fails to disclose receiving an input to modify a configuration of the first camera and the second camera.
Park disclose receiving an input to modify a configuration of the first camera and the second camera (paragraph [0071]: the position of depth camera sensor 175 can be changed by the user (e.g., by rotating the lens and the sensor in the device housing) so that a depth camera sensor 175 is used along with the touch screen display for both video conferencing and still and/or video image acquisition; paragraph [0322]: members of the home (e.g., have rights to modify settings of accessories corresponding to the home), camera user interface object 802b for configuring settings of camera accessories that are part of the home automation system).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large and Luskin’s to configure camera as taught by Park, to obtain desired images.
Regarding claim 3, Lee as modified by Large, Luskin and Park discloses the method of claim 2, further comprising:
in response to receiving the input, generating an updated AR output by using the second video stream as the primary input to the first AR experience bundle and using the first video stream as the secondary input to the first AR experience bundle; and causing presentation of the updated AR output on a display of the client device (Park’s paragraph [0293]: the camera view and the second camera view are horizontally scrollable (e.g., move horizontally in unison) via user input (e.g., to scroll into view additional camera views of other cameras while scrolling out of view the camera view and the second camera view); paragraph [0226]: in FIG. 6G, concurrently horizontally scrolls the objects in cameras region 616 and changes the focus to camera preview 616b of the camera located in the back yard of the home; paragraph [0439]: At FIG. 10C... third camera view UI object 1012a is visually emphasized to indicate the focus (as shown in FIG. 10C via the bold border of third camera view UI object 1012a)).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large and Luskin’s to configure camera as taught by Park, to obtain desired images.
Regarding claim 4, Shirmohamadi as modified by Borremans and Park discloses the method of claim 2, wherein the primary input is presented at a first display position within the AR output and the secondary input is presented at a second display position within the AR output, the method further comprising:
in response to receiving the input, generating an updated AR output wherein the primary input is presented at the second display position within the AR output and the secondary input is presented at the first position within the AR output; and causing presentation of the updated AR output on a display of the client device (Park’s paragraph [0293]: the camera view and the second camera view are horizontally scrollable (e.g., move horizontally in unison) via user input (e.g., to scroll into view additional camera views of other cameras while scrolling out of view the camera view and the second camera view); paragraph [0226]: in FIG. 6G, concurrently horizontally scrolls the objects in cameras region 616 and changes the focus to camera preview 616b of the camera located in the back yard of the home; paragraph [0439]: At FIG. 10C... third camera view UI object 1012a is visually emphasized to indicate the focus (as shown in FIG. 10C via the bold border of third camera view UI object 1012a)).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large and Luskin’s to configure camera as taught by Park, to obtain desired images.
Claim 5-7 are rejected under 35 U.S.C. 103 as being unpatentable over Lee U.S. Patent Application 20190037173 in view of Large U.S. Patent Application 20210144313, in view of Luskin U.S. Patent Application 20050030379, and further in view of Shirmohamadi U.S. Patent Application 20200120269.
Regarding claim 5, Lee as modified by Large and Luskin discloses all the features with respect to claim 1 as outlined above. However, Lee as modified by Large and Luskin fails to disclose generating a cutout of facial elements depicted in images obtained from the first camera; and superimposing, on top of an image obtained from the second camera, the cutout of the facial elements depicted in the images obtained from the first camera.
Shirmohamadi discloses generating a cutout of facial elements depicted in images obtained from the first camera; and superimposing, on top of an image obtained from the second camera, the cutout of the facial elements depicted in the images obtained from the first camera (paragraph [0041]: either automatically, or at the direction of the user, facilitating the placement and/or sizing of the selected portion of the second image so that it is placed within the first image, to create a combined image; paragraph [0027]: Both images are displayed simultaneously onto phone's single display... The whole or a portion of the second image may then be positioned over part of the first image; paragraph [0032]: a portion of the second image (the ‘image portion’), usually a face, is selected and marked out on the second image. The selected portion of the second image may then be positioned over the first image).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large and Luskin’s to overlay images as taught by Shirmohamadi, to obtain improved selfies.
Regarding claim 6, Lee as modified by Large, Luskin and Shirmohamadi discloses the method of claim 1, further comprising:
overlaying one or more facial features of a person depicted in the first video on a first real-world object depicted in the second video (Shirmohamadi’s paragraph [0041]: either automatically, or at the direction of the user, facilitating the placement and/or sizing of the selected portion of the second image so that it is placed within the first image, to create a combined image; paragraph [0027]: Both images are displayed simultaneously onto phone's single display... The whole or a portion of the second image may then be positioned over part of the first image; see fig. 4-5; paragraph [0032]: a portion of the second image (the ‘image portion’), usually a face, is selected and marked out on the second image. The selected portion of the second image may then be positioned over the first image);
receiving input that selects a second real-world object in the second video by tapping a region at which the second real-world object is displayed; and in response to receiving the input, moving the one or more facial features from being overlaid on the first real-world object in the second video to being overlaid on the second real-world object in the second video (Shirmohamadi’s paragraph [0032]: The user may select the marked-out image portion. This may be done by touching or tapping the screen at the appropriate position (on the selected image) and the user may then place the selected image portion from the second image at a location in the first displayed image by tapping the desired location or dragging the image from the panel displaying the second image to/into the panel displaying the first image).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large and Luskin’s to overlay images as taught by Shirmohamadi, to obtain improved selfies.
Regarding claim 7, Lee as modified by Large, Luskin and Shirmohamadi discloses the method of claim 1, further comprising:
receiving an input to initiate the first AR bundle while at least one of the first camera and the second camera is not capturing video; and in response to receiving the input, causing the first camera to capture the first video stream and the second camera to capture the second video stream, wherein a face depicted in an image obtained by the first camera is embedded into a real-world object depicted in image obtained by the second camera (Shirmohamadi's paragraph [0041]: a screen disposed on one side of the cell-phone, and at least one control unit on the cell-phone for controlling functions of the cell-phone… a computer processing unit (CPU) programmed to instruct the cell-phone to perform certain steps, including, but not limited to: (a) activating both first and second cameras so both are active simultaneously, and are able to communicate an electronic first image and an electronic second image to the CPU… (c) responding to the activation of a control unit which acts as a shutter release control, such that two images (one from the front and one from the back camera) are substantially simultaneously captured; paragraph [0032]: a portion of the second image (the ‘image portion’), usually a face, is selected and marked out on the second image. The selected portion of the second image may then be positioned over the first image; Luskin’s paragraph [0040]: the default mode can be set to a full-sized rear view and two small windows for left and right cameras. For a system with four cameras, this means that the default mode for left and right cameras are `Window`, the default mode for rear view is `Full` and the default for front camera is `Off`; Lee’s paragraph [0790]: The user of the first electronic device may select (e.g., touch or tap) a dual camera object 5110 (e.g., dual camera icon) for executing the dual camera, or the first electronic device may execute the narration mode in response to a user's voice input. When the narration mode is executed, the first electronic device may drive the front camera simultaneously in a state in which the rear camera is activated).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large and Luskin’s to overlay images as taught by Shirmohamadi, to obtain improved selfies.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Lee U.S. Patent Application 20190037173 in view of Large U.S. Patent Application 20210144313, in view of Luskin U.S. Patent Application 20050030379, in view of Shirmohamadi U.S. Patent Application 20200120269, and further in view of Jorasch U.S. Patent Application 20210399911.
Regarding claim 9, Lee as modified by Large, Luskin and Shirmohamadi discloses all the features with respect to claim 1 as outlined above. However, Lee as modified by Large, Luskin and Shirmohamadi fails to disclose identifying a first body part of a first person; identifying a second body part of the first person; and generating a combined image depicting a first AR element overlaid on the first body part and a second AR element overlaid on the second body part.
Jorasch discloses identifying a first body part of a first person; identifying a second body part of the first person; and generating a combined image depicting a first AR element overlaid on the first body part and a second AR element overlaid on the second body part (paragraph [1143]: The central controller could interpret head movements and provide a visual overlay of these movements in video conferencing software. For instance, the central controller could interpret a head nod and overlay a “thumbs up” symbol. If the central controller detects an emotional reaction, it could overlay an emoji. These overlays could provide visual cues to meeting participants about the group's opinion at a given moment; paragraph [2835]: the user could preselect the smile type or disgruntled face they would like to use when the camera determines these emotions. This would be displayed on the avatar for all to see).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large, Luskin and Shirmohamadi’s to use avatar as taught by Jorasch, to attend virtual meeting efficiently.
Claim 11-13 are rejected under 35 U.S.C. 103 as being unpatentable over Lee U.S. Patent Application 20190037173 in view of Large U.S. Patent Application 20210144313, in view of Luskin U.S. Patent Application 20050030379, in view of Shirmohamadi U.S. Patent Application 20200120269, and further in view of Jovanovic U.S. Patent Application 20210200942.
Regarding claim 11, Lee as modified by Large, Luskin and Shirmohamadi discloses causing the first camera to capture the first video stream and the second camera to capture the second video stream; and displaying images included in the first video stream and second video stream on the display of the device according to the first display layout (Shirmohamadi’s paragraph [0041]: a computer processing unit (CPU) programmed to instruct the cell-phone to perform certain steps, including, but not limited to: (a) activating both first and second cameras so both are active simultaneously, and are able to communicate an electronic first image and an electronic second image to the CPU… (c) responding to the activation of a control unit which acts as a shutter release control, such that two images (one from the front and one from the back camera) are substantially simultaneously captured… either automatically, or at the direction of the user, facilitating the placement and/or sizing of the selected portion of the second image so that it is placed within the first image, to create a combined image). However, Lee as modified by Large, Luskin and Shirmohamadi fails to disclose in response to receiving the input, displaying a plurality of display layouts; receiving a selection of a first layout from the plurality of layouts.
Jovanovic discloses in response to receiving the input, displaying a plurality of display layouts; receiving a selection of a first layout from the plurality of layouts (paragraph [0104]: use the ranking to select one or more recommended alternative layouts from the plurality of alternative layouts; and provide the one or more recommended alternative layouts for display on a device).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large, Luskin and Shirmohamadi’s to select layout as taught by Jovanovic, to automatically enhance user content by dynamically generating alternative layouts given an original layout of the user content.
Regarding claim 12, Lee as modified by Large, Luskin, Shirmohamadi and Jovanovic discloses the method of claim 11, wherein the plurality of display layouts comprises at least one of:
a horizontal layout in which in images included in the first video stream are displayed on a top portion of a screen and the images included in the second video stream displayed on a bottom portion of the screen;
a vertical layout in which images included in the first video stream are displayed on a left portion of the screen and images included in the second video stream are displayed on a right portion of the screen;
a picture-in-picture (PIP) layout in which images included in the first video stream are displayed in full screen and images included in the second video stream are displayed within the full screen;
a green screen layout in which a portion of images included in the first video stream are superimposed on a portion of images included in the second video stream based on one or more object segmentation models (Shirmohamadi’s paragraph [0041]: either automatically, or at the direction of the user, facilitating the placement and/or sizing of the selected portion of the second image so that it is placed within the first image, to create a combined image; paragraph [0032]: a portion of the second image (the ‘image portion’), usually a face, is selected and marked out on the second image. The selected portion of the second image may then be positioned over the first image).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large, Luskin and Shirmohamadi’s to select layout as taught by Jovanovic, to automatically enhance user content by dynamically generating alternative layouts given an original layout of the user content.
Regarding claim 13, Lee as modified by Large, Luskin, Shirmohamadi and Jovanovic discloses the method of claim 11, wherein the plurality of display layouts comprises a face builder layout in which cutouts of facial elements depicted in the images included in the first video stream are superimposed on top of images included in the second video stream (Shirmohamadi’s paragraph [0041]: either automatically, or at the direction of the user, facilitating the placement and/or sizing of the selected portion of the second image so that it is placed within the first image, to create a combined image; paragraph [0032]: a portion of the second image (the ‘image portion’), usually a face, is selected and marked out on the second image. The selected portion of the second image may then be positioned over the first image).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large, Luskin and Shirmohamadi’s to select layout as taught by Jovanovic, to automatically enhance user content by dynamically generating alternative layouts given an original layout of the user content.
Claim 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Lee U.S. Patent Application 20190037173 in view of Large U.S. Patent Application 20210144313, in view of Luskin U.S. Patent Application 20050030379, in view of Shirmohamadi U.S. Patent Application 20200120269, and further in view of Morin U.S. Patent Application 20090202114.
Regarding claim 14, Lee as modified by Large, Luskin and Shirmohamadi discloses detecting a real-world object depicted in a first image included in the primary video stream (Shirmohamadi’s paragraph [0032]: a portion of the second image (the ‘image portion’), usually a face, is selected and marked out on the second image. The selected portion of the second image may then be positioned over the first image). However, Lee as modified by Large, Luskin and Shirmohamadi fails to disclose extracting one or more textures from the real-world object depicted in the first image; selecting a target object depicted in a second image included in the secondary video stream; and generating an AR element comprising the target object modified based on the one or more textures extracted from the real-world object depicted in the first image, the AR output including the AR element within the second image.
Morin discloses extracting one or more textures from the real-world object depicted in the first image (paragraph [0119]: Reverse rendering is a process of extracting a user face texture from a video feed so that the texture can be applied on another object or media, such as an avatar, movie character);
selecting a target object depicted in a second image included in the secondary video stream; and generating an AR element comprising the target object modified based on the one or more textures extracted from the real-world object depicted in the first image, the AR output including the AR element within the second image (paragraph [0043]: the user may apply a different head and body around the facial area of their character; paragraph [0158]: users of the gaming devices 558, 560 may have real-time video capture at their respective locations, and may have the captured video transmitted, perhaps in altered or augmented form, to the other gaming device to improve the quality of gameplay; paragraph [0203]: When the face texture is applied to a face in a movie, it may allow a user the ability to change the facial expressions of the actors; paragraph [0194]: FIG. 7C illustrates an example of other games that can be implemented with captured video. In frame 718, a poker game is illustrated. One or more faces can be added corresponding to the different players in the game).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large, Luskin and Shirmohamadi’s to extract texture as taught by Morin, to provide live action image capture.
Regarding claim 15, Lee as modified by Large, Luskin, Shirmohamadi and Morin discloses the method of claim 14, wherein the one or more textures depicts a face of a first user and the target object is a second user, the AR element depicting the second user with the face of the first user in place of a face of the second user (Morin’s paragraph [0119]: Reverse rendering is a process of extracting a user face texture from a video feed so that the texture can be applied on another object or media, such as an avatar, movie character; paragraph [0158]: users of the gaming devices 558, 560 may have real-time video capture at their respective locations, and may have the captured video transmitted, perhaps in altered or augmented form, to the other gaming device to improve the quality of gameplay; paragraph [0043]: the user may apply a different head and body around the facial area of their character).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large, Luskin and Shirmohamadi’s to extract texture as taught by Morin, to provide live action image capture.
Regarding claim 16, Lee as modified by Large, Luskin, Shirmohamadi and Morin discloses the method of claim 15, wherein generating the AR output further comprises:
extracting one or more textures depicting the face of the second user depicted in the second image; and generating a second AR element depicting the first user with the face of the second user in place of the face of the first user, the AR output including both the AR element and the second AR element (Morin’s paragraph [0119]: Reverse rendering is a process of extracting a user face texture from a video feed so that the texture can be applied on another object or media, such as an avatar, movie character; paragraph [0158]: users of the gaming devices 558, 560 may have real-time video capture at their respective locations, and may have the captured video transmitted, perhaps in altered or augmented form, to the other gaming device to improve the quality of gameplay; paragraph [0043]: the user may apply a different head and body around the facial area of their character).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Lee, Large, Luskin and Shirmohamadi’s to extract texture as taught by Morin, to provide live action image capture.
Allowable Subject Matter
Claim 17 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims, and overcome the double patenting rejection.
The following is a statement of reasons for the indication of allowable subject matter:
Claim 17 is about generating a first AR experience bundle by: presenting an AR experience development user interface for configuring the first AR experience bundle; and presenting a prompt in the AR experience development user interface that comprises an option for enabling simultaneous activation of multiple image sensors on a client device, wherein configuration data for the first AR experience bundle is updated to indicate that simultaneous activation of the multiple image sensors has been enabled in response to receiving input that selects the option.
Lee 20190037173, Large 20210144313, Luskin 20050030379, Park 20210385417, Shirmohamadi 20200120269 and Borremans 20220006994 combined cannot teach these features perfectly. Although the individual limitations recited in the claims do exist in isolation in the prior art of record, the specific combination recited by the Applicant is found to be non-obvious. These limitations when read in light of the rest of the limitations in the claim make the claim allowable subject matter.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Yi Yang whose telephone number is (571)272-9589. The examiner can normally be reached on Monday-Friday 9:00 AM-6:00 PM EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached on 571-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/YI YANG/
Primary Examiner, Art Unit 2616
Prosecution Insights