DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 7/10/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Rejections under 35 USC 112(b) and 112(d) are withdrawn.
Applicant's arguments regarding Prior Art rejections have been fully considered but they are not persuasive.
On Page 9 of Remarks, in the first Paragraph, Applicant argues, “Ge does not describe or suggest at least the feature of "image data of a hand in a real-world scene, the image data comprising a plurality of pixels including a plurality of foreground pixels corresponding to the hand," as recited in the amended claims. In contrast, Ge, at Paragraph [0016], describes that "[t]he 3D hand mesh is generated directly from a single RGB image depicting a real- world hand and represents the pose (e.g., joint locations) and shape (e.g., surface features) of the hand that is depicted in the RGB image. The disclosed embodiments generate the 3D hand mesh without also obtaining a depth map of the real-world hand."”
Ge expressly discloses the above referenced feature. In fact, the excerpt from Ge provided above confirms this (see bolded phrase above). An RGB image of a real-world hand, taught by Ge, amounts to image data of a hand in a real-world scene, with foreground pixels (pixels of the hand). See Ge, Figure 6 element 601 for additional support.
In the next Paragraph on Page 9 of Remarks, Applicant argues, “Ge also does not describe or suggest at least the feature of "the respective depth value for each foreground pixel being measured from a specified landmark of the hand, the estimated depth data defining a 3D point cloud of the hand in the real-world scene providing a reconstruction of the hand in the real-world scene," as recited in the amended claims. In contrast, Ge, at Paragraph [0016], describes that "image features of a single RGB image are extracted by one or more machine learning techniques and then graph convolutions (e.g., using a graph CNN) are applied hierarchically with upsampling and nonlinear activations to generate 3D hand mesh vertices." Ge further describes, at Paragraph [0069], that "the graph CNN module 418 generates 3D coordinates of vertices in the hand mesh and estimates the 3D hand pose from the mesh."”
Firstly, note that the respective depth values of Ge correspond to vertices, as described in the above bolded segment. These vertices amount to landmarks. Accordingly, the respective depth value for each foreground (hand) pixel is measured from a specific landmark (vertices). The above claim limitation, as currently worded, does not require that a landmark be treated as an origin point from which all coordinates are measured relative to. While this appears to be the Applicant’s intention of the amendment, the current wording does not require this narrow interpretation. Next, Ge does teach the “the estimated depth data defining a 3D point cloud of the hand in the real-world scene providing a reconstruction of the hand in the real-world scene” in 8 element 803 (Output 3D Mesh).
The remaining paragraphs on Page 9 argue that the other references do not provide the features not found in Ge, and that the dependent claims are allowable as dependent on the independent claims 1, 8, and 15.
These arguments are moot in view of the above arguments.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 8, and 15, and 21-23 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ge (US 20200184721 A1).
Regarding claim 1, Ge teaches, “A computer-implemented method comprising: capturing, by one or more processors using a camera of an eXtended Reality (XR) system, image data of a hand in a real-world scene, the image data comprising a plurality of pixels including a plurality of foreground pixels corresponding to the hand;” (Ge, Paragraph 16, “The disclosed embodiments improve the efficiency of using the electronic device by applying machine learning techniques, including a graph CNN, to generate a 3D hand mesh for presentation in a VR or AR application. The 3D hand mesh is generated directly from a single RGB image depicting a real-world hand and represents the pose (e.g., joint locations) and shape (e.g., surface features) of the hand that is depicted in the RGB image. The disclosed embodiments generate the 3D hand mesh without also obtaining a depth map of the real-world hand. Specifically, according to the disclosed techniques, image features of a single RGB image are extracted by one or more machine learning techniques and then graph convolutions (e.g., using a graph CNN) are applied hierarchically with upsampling and nonlinear activations to generate 3D hand mesh vertices. According to the disclosed embodiments, the generated 3D hand mesh can better represent highly variable 3D hand shapes (e.g., surface features) and their local details. This enables a user device with a simple RGB camera (without a depth camera) to accurately and quickly render an animated 3D hand model of the real-world user's hand within the VR or AR application, allowing the user to interact with the VR or AR content in a more realistic environment.“ Note that the hand pixels amount to the foreground pixels corresponding to the hand.)
“generating, by the one or more processors, using the image data and a depth estimation model trained using synthetic 2D image data, estimated depth data comprising a respective depth value for each foreground pixel of the plurality of foreground pixels of the image data corresponding to the hand, the respective depth value for each foreground pixel being measured from a specified landmark of the hand, the estimated depth data defining a 3D point cloud of the hand in the real-world scene that provides a reconstruction of the hand in the real-world scene;” (Ge, Figure 6 element 603 describes the estimation of depth data of the real hand (3D hand mesh). The depth estimation model trained using synthetic 2D image data is disclosed in Figure 7 and Paragraph 83, “At operation 701, the hand shape and pose estimation system 124 obtains a first plurality of input images that include synthetic representations of a hand. For example, machine learning techniques network 410 initially receives synthetic hand training image data 402. An illustrative synthetic hand training image data 801 and its corresponding output is shown in a first row 810 of FIG. 8.” Further, see Paragraph 69 for description of 3D coordinates of vertices in the hand mesh: “An embodiment of the graph CNN module 418 is shown in FIG. 5. Particularly, the graph CNN module 418 generates 3D coordinates of vertices in the hand mesh and estimates the 3D hand pose from the mesh. In this way, the graph CNN module 418 models, based on features extracted by other machine learning technique modules of FIG. 5, a post of a hand depicted in a monocular image by adjusting skeletal joint positions of a 3D hand mesh and also models a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image. The resulting 3D hand mesh is then generated for display.” Note that the foreground pixels of the hand are respectively measured from landmarks of the hand (vertices). Additionally, as explained in the Arguments above, Ge expressly discloses “the estimated depth data defining a 3D point cloud of the hand in the real-world scene providing a reconstruction of the hand in the real-world scene” in 8 element 803.)
“generating, by the one or more processors, an XR effect using the estimated depth data and the image data; and providing, by the one or more processors, the XR effect to a user in a user interface.” (Ge, Figure 6, elements 604 and 605).
Regarding claim 21, Ge teaches, “The computer-implemented method of claim 1,
“wherein the estimated depth data further comprises an estimated segmentation mask of the hand, the estimated segmentation mask comprising a respective segmentation value for each pixel of the plurality of pixels,” (Ge, Figure 8 element 803 shows the estimated depth data, wherein only the hand is reconstructed. This amounts to to a mask for the hand wherein the non-hand pixels correspond to a ‘non-hand’ segmentation (segmentation mask) and the hand pixels correspond to a ‘hand’ segmentation (the claimed estimated segmentation mask). The segmentation values amount to pixel coordinates of the hand segmentation.)
“and wherein the depth estimation model is trained based on a loss function using a set of depths and a segmentation mask paired with a synthetic 2D image of the synthetic 2D image data.” (Ge, Paragraph 74 and Figure 4, “In some implementations, in a first training phase, synthetic hand training image data 402 is used to train the stacked hourglass network (e.g., first machine learning technique module 412) and the 3D pose regressor (e.g., another machine learning technique) separately with the heat-map loss module 414 and the 3D pose loss module 432, respectively. Subsequently, the stacked hourglass network, the residual network (e.g., the second machine learning technique module 416), and the graph CNN module 418 are trained for mesh generation with the combined loss function of Equation 17:
PNG
media_image1.png
42
519
media_image1.png
Greyscale
In some implementations, λH=0.5, λM=1, λJ=1. Particularly, in the first training phase, each of the loss modules 414, 420 and 432 is provided with ground truth information corresponding to a given image from synthetic hand training image data 402 being processed. The first machine learning technique module 412 processes the given image from synthetic hand training image data 402 to generate an estimated heat map from the image, the graph CNN module 418 processes the given image to estimate a hand mesh, and the pose regressor module 430 processes the given image to estimate a 3D pose graph. Each of these networks is trained to minimize the corresponding loss of loss modules 414, 420 and 432.” The set of depths and a segmentation mask amount to the hand mesh and estimated 3D pose graph, which include depth values (set of depths) of hand region (segmentation mask).)
Regarding claims 8, 15, 22 and 23, claims 8 and 22 recite a system, with one or more processors and a memory, with elements corresponding to the steps recited in claims 1 and 21. Therefore, the recited elements of this claim is mapped to the analogous steps in the corresponding method claims. Claims 15 and 23 recite a non-transitory machine-readable storage medium storing a program with instructions corresponding to the steps recited in claims 1 and 21. Therefore, the recited programming instructions are mapped to the analogous steps in the corresponding method claim. Additionally, Ge teaches a system with one or more processors and a memory, and a non-transitory machine-readable storage medium (Ge, Paragraph 109, ““MACHINE-READABLE MEDIUM” in this context refers to a component, device, or other tangible media able to store instructions and data temporarily or permanently and may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage (e.g., Erasable Programmable Read-Only Memory (EEPROM)) and/or any suitable combination thereof. The term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store instructions. The term “machine-readable medium” shall also be taken to include any medium, or combination of multiple media, that is capable of storing instructions (e.g., code) for execution by a machine, such that the instructions, when executed by one or more processors of the machine, cause the machine to perform any one or more of the methodologies described herein. Accordingly, a “machine-readable medium” refers to a single storage apparatus or device, as well as “cloud-based” storage systems or storage networks that include multiple storage apparatus or devices. The term “machine-readable medium” excludes signals per se.”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 9, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ge in view of Xia (CN107992858A)
Regarding claim 2, Ge teaches, “The computer-implemented method of claim 1,”
Ge does not expressly disclose, “wherein generating the estimated depth data comprises: determining, by the one or more processors, cropping boundary data using the image data and a detection model; and cropping, by the one or more processors, the image data using the cropping boundary data.”
Xia discloses “wherein generating the estimated depth data comprises: determining, by the one or more processors, cropping boundary data using the image data and a detection model; and cropping, by the one or more processors, the image data using the cropping boundary data.” (Xia, 2nd to last Paragraph of Page 2, and second Paragraph of page 3, “wherein the three-dimensional gesture estimation method of real-time, firstly as input color image, using the most advanced detector to locate the hand for each hand image in the input, hand detector generating a likelihood estimation value and the hand bounding box coordinates, pre-training network running OpenPose (body language recognition system) to estimate the two-dimensional position of the hand joint to generate hand 2D joint position of the hotspot graph, finally, nonlinear minimization of hand fitting the 3D model to the estimated 2D joint position. recovering the 3D hand gesture.”; “wherein, said 2D joint estimation according to the detected boundary frame cropping the image, the 2D fed to the key point detector, because training only on the left hand key point detector, passes along the Y-axis mirror image processing right, the output of the detector is 21 hotspot graph, corresponding to the 20 key points (four) per finger and a wrist point estimate.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use the detection model of Xia to generate the bounding box and perform the cropping of the images of Ge using the processor of Ge.
The motivation for doing so would have been to isolate the hand from an image which contains non-hand objects to avoid inadvertent image processing on the non-hand objects. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Ge with the above bounding box generation and cropping of Xia to obtain the invention of claim 2.
Regarding claims 9 and 16, claim 9 recites a system, with one or more processors and a memory, with elements corresponding to the steps recited in claim 2. Therefore, the recited elements of this claim are mapped to the analogous steps in the corresponding method claim. Claim 16 recites a non-transitory machine-readable storage medium storing a program with instructions corresponding to the steps recited in claim 2. Therefore, the recited programming instructions are mapped to the analogous steps in the corresponding method claim. The rationale and motivation to combine the Ge and Xia references apply here. Additionally, Ge teaches a system with one or more processors and a memory, and a non-transitory machine-readable storage medium (Ge, Paragraph 109, ““MACHINE-READABLE MEDIUM” in this context refers to a component, device, or other tangible media able to store instructions and data temporarily or permanently and may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage (e.g., Erasable Programmable Read-Only Memory (EEPROM)) and/or any suitable combination thereof. The term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store instructions. The term “machine-readable medium” shall also be taken to include any medium, or combination of multiple media, that is capable of storing instructions (e.g., code) for execution by a machine, such that the instructions, when executed by one or more processors of the machine, cause the machine to perform any one or more of the methodologies described herein. Accordingly, a “machine-readable medium” refers to a single storage apparatus or device, as well as “cloud-based” storage systems or storage networks that include multiple storage apparatus or devices. The term “machine-readable medium” excludes signals per se.”)
Claim(s) 4-7, 11-14, and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ge in view of Guay (US20210374993A1) further in view of Hushchyn (US20180300927A1)
Regarding claim 4, Ge teaches, “The computer-implemented method of claim 1,”
While Ge teaches that training the depth estimation model comprises receiving 3D data of a measured hand;” (Ge, Paragraph 41, “In some implementations, the second plurality of images, stored in synthetic and real hand training images 209, includes hand RGB images with the corresponding depth images of each RGB image captured using a RGB-D camera (e.g., an RGB camera fitted with a depth sensor or depth camera.)”); and Ge teaches training “the depth estimation model using the synthetic 2D image data and the target depth data”, (Ge, Paragraph 86, “At operation 704, the hand shape and pose estimation system 124 trains the first and second machine learning techniques together with a graph CNN based on the first plurality of input images. For example, for a given image from the synthetic hand training image data 402, ground truth hand mesh information is obtained and provided to hand mesh loss module 420. The graph CNN module 418 estimates a hand mesh for the given image and is trained to minimize the loss computed by hand mesh loss module 420. The machine learning techniques network 410 are trained to minimize the loss together in accordance with Equation 17.”), Ge does not expressly disclose using the 3D data of the measured hand to generate both the synthetic 2D image data and target depth data that are used for training, and the use of a second processor.
Guay teaches the generation of 3D model data using measured 3D data (Guay, Paragraph 43, “As shown, the inference subsystem 104 inputs the input image 188 into the pose estimation model 160 and, in response, the pose estimation model 160 outputs the estimated 3D pose 190. The estimated 3D pose 190 is an estimate of the 3D pose of the costumed person 122(2) as depicted in the input image 188. The estimated 3D pose 190 specifies explicitly or implicitly, in any technically feasible fashion, and without limitation, a set of 3D positions associated with a set of joints. In some embodiments, the set of 3D positions estimates positions and orientations associated with a set of physical joints of the costumed person 122(2).”; generating synthetic 2D images using the generated 3D model data (Guay, Paragraph 93, “As shown, a method 600 begins a step 602, where the training application 140 acquires the pose dataset 130 and the 3D model 124 of the costumed person 122(1). At step 604, for each of the 3D poses 132 included in the pose dataset 130, the variation engine 240 generates any number of the image recipes 262 based on the synthesis variation list 134. At step 606, the synthesis/rendering engine 270 generates the synthetic images 278 based on the image recipes 262 and the 3D model 124.”); and generating target depth data by pairing depths with the synthetic 2D images using the synthetic 2D image data and the 3D model data; (Guay, Paragraphs 94 and 98, “At step 608, the synthesis/rendering engine 270 generates the synthetic training dataset 280 based on the synthetic images 278 and the associated 3D poses 132. At step 610, the training application 140 trains the machine-learning model 290 based on the synthetic training dataset 280 to generate the pose estimation model 160. At step 612, the training application 140 transmits the pose estimation model 160 to the inference subsystem 104.”; “During a training phase, the variation engine generates multiple variants based on the list of synthesis variations. Each variant specifies a different combination of body changes, lighting, camera viewpoint, and texture. For each of the 3D poses, the variation engine generates multiple image recipes, where each image recipe specifies the 3D pose and a different variant. For each of the image recipes, the synthesis/rendering engine optionally modifies the 3D model as per any specified body changes and then fits the associated 3D pose to either the 3D model or the modified 3D model to generate a posed 3D model. For each of the image recipes, the synthesis/rendering engine renders the associated posed 3D model based on the specified lighting, camera viewport, and texture to generate a synthetic image. For each synthetic image, the synthesis/rendering engine generates a training item that includes the synthetic image and the associated 3D pose. Together, the training items form a synthetic training dataset. Subsequently, the training application trains the machine-learning model based on the synthetic training database to generate the pose estimation model.”)
Hushchyn teaches using a second processor for training (Hushchyn, Paragraph 11, “In some embodiments, the training of the at least one face detection regressor is performed by at least one second processor; and where the at least one second processor is distinct from the at least one first processor of the portable electronic device and is remotely located from the portable electronic device.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to generate the synthetic images of Ge from the 3D hand images of Ge using the above described process of Guay, including 3D model generation from the 3D image and synthetic image generation from the 3D model. It would have further been obvious to incorporate these synthetic images paired with associated 3D pose information, performed by Guay above, as the training data of Ge. Additionally, it would have been obvious to use a second processor, taught by Hushchyn, for the training method of Ge in view of Guay.
The motivation for incorporating Guay would have been to generate more accurate training data using actual images, rather than fully computer-generated images, and further train with the more accurate training data for a more accurate depth estimation model. The motivation for incorporating Hushchyn would have been to enable parallel processing for simultaneous training and model generation. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Ge with the above teachings of Guay and Hushchyn to obtain the above limitations.
Accordingly, Ge in view of Guay further in view of Hushchyn fully disclose “receiving, by second one or more processors, 3D data of a measured hand; generating, by the second one or more processors, 3D model data of the measured hand using the 3D data; generating, by the second one or more processors, synthetic 2D image data comprising one or more synthetic 2D images using the 3D model data; generating, by the second one or more processors, target depth data comprising one or more sets of depths paired to the one or more synthetic 2D images using the synthetic 2D image data and the 3D model data; training, by the second one or more processors, the depth estimation model using the synthetic 2D image data and the target depth data.”
Regarding claim 5, Ge in view of Guay further in view of Hushchyn teach “The computer-implemented method of claim 4,”
“wherein generating the synthetic 2D image data comprises using camera and lighting parameter data.” (Ge, Paragraph 40, In some implementations, the first plurality of images (also referred to as synthetic images), stored in synthetic and real hand training images 209, provides the labels of both 3D hand joint locations and full 3D hand meshes. A 3D hand model is generated, rigged with joints, and then photorealistic textures are applied on the 3D hand model as well as natural lighting using high-dynamic range (HDR) images. The variations of the hand are modeled by creating blend shapes with different shapes and ratios, and then random weights are applied to the blend shapes. Hand poses from 500 common hand gestures and 1000 unique camera viewpoints are created and captured in the first plurality of images. To simulate real-world diversity, 30 lightings and five skin colors are used. The hand is rendered using global illumination. In some implementations, the first plurality of images includes 375,000 hand RGB images with large variations. In some embodiments, only a portion (e.g., 315,000) of the first plurality of images are used in the first training phase to train the machine learning techniques. During training or before, each rendered hand in the first plurality of images is cropped from the image and blended with a randomly selected background image (e.g., a city image, a living room image, or any other suitable image obtained randomly or pseudo-randomly from a background image server(s)).”)
Regarding claim 6, Ge in view of Guay further in view of Hushchyn teach “The computer-implemented method of claim 5,”
Ge in view of Guay further in view of Hushchyn do not expressly disclose “wherein the camera and lighting parameter data comprise randomized values.”
Guay further discloses, “wherein the camera and lighting parameter data comprise randomized values.” (Guay, Paragraph 73, “In some alternate embodiments, the training application 140 generates the synthetic training dataset 280 in any technically feasible fashion based on the 3D model 124 and any number of 3D poses 132. In the same or other alternate embodiments, the training application 140 ensures that the synthetic training dataset 280 represents variations in any number and/or types of depicted objects (e.g., the costumed person 122) and/or conditions in any technically feasible fashion. For instance, in some alternate embodiments, the training application 140 directly and/or indirectly performs any number of domain randomization operations to create variations in body shape, lighting, camera viewpoint, and texture across the synthetic images 278 and therefore across the synthetic training dataset 280.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to perform the randomization of camera and lighting parameters, further taught by Guay, into the synthetic image generation of Ge in view of Guay further in view of Hushchyn.
The motivation for doing so would have been to train the model with a greater variety of training images for enhanced performance in a greater variety of real world circumstances. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Ge in view of Guay further in view of Hushchyn with the above additional teaching of Guay to achieve the invention of claim 6.
Regarding claim 7, Ge in view of Guay further in view of Hushchyn teach “The computer-implemented method of claim 4,”
“wherein training the depth estimation model further comprises: determining, by the second one or more processors, cropping boundary data using the synthetic 2D image data and a detection model; and cropping, by the second one or more processors, the synthetic 2D image data using the cropping boundary data.” (Ge, Paragraph 40, “In some implementations, the first plurality of images (also referred to as synthetic images), stored in synthetic and real hand training images 209, provides the labels of both 3D hand joint locations and full 3D hand meshes. A 3D hand model is generated, rigged with joints, and then photorealistic textures are applied on the 3D hand model as well as natural lighting using high-dynamic range (HDR) images. The variations of the hand are modeled by creating blend shapes with different shapes and ratios, and then random weights are applied to the blend shapes. Hand poses from 500 common hand gestures and 1000 unique camera viewpoints are created and captured in the first plurality of images. To simulate real-world diversity, 30 lightings and five skin colors are used. The hand is rendered using global illumination. In some implementations, the first plurality of images includes 375,000 hand RGB images with large variations. In some embodiments, only a portion (e.g., 315,000) of the first plurality of images are used in the first training phase to train the machine learning techniques. During training or before, each rendered hand in the first plurality of images is cropped from the image and blended with a randomly selected background image (e.g., a city image, a living room image, or any other suitable image obtained randomly or pseudo-randomly from a background image server(s)”; Additionally, note that as the references are combined in claim 4, the entire training process performed on the second processor. Therefore, the elements of claim 7, which are part of the training process, are performed on the second processor.)
Regarding claims 11-14, and 18-20, claims 11-14 recite a system, with one or more processors and a memory, with elements corresponding to the steps recited in claims 4-7. Therefore, the recited elements of these claims are mapped to the analogous steps in the corresponding method claims. Claims 18-20 recite a non-transitory machine-readable storage medium storing a program with instructions corresponding to the steps recited in claims 4, 5, and 7. Therefore, the recited programming instructions are mapped to the analogous steps in the corresponding method claims. The rationale and motivation to combine the Ge, Guay, and Hushchyn references apply here. Additionally, Ge teaches a system with one or more processors and a memory, and a non-transitory machine-readable storage medium (Ge, Paragraph 109, ““MACHINE-READABLE MEDIUM” in this context refers to a component, device, or other tangible media able to store instructions and data temporarily or permanently and may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage (e.g., Erasable Programmable Read-Only Memory (EEPROM)) and/or any suitable combination thereof. The term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store instructions. The term “machine-readable medium” shall also be taken to include any medium, or combination of multiple media, that is capable of storing instructions (e.g., code) for execution by a machine, such that the instructions, when executed by one or more processors of the machine, cause the machine to perform any one or more of the methodologies described herein. Accordingly, a “machine-readable medium” refers to a single storage apparatus or device, as well as “cloud-based” storage systems or storage networks that include multiple storage apparatus or devices. The term “machine-readable medium” excludes signals per se.”)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON JOSEPH SORRIN whose telephone number is (703)756-1565. The examiner can normally be reached Monday - Friday 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON JOSEPH SORRIN/
Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights