DETAILED ACTION
*Note in the following document:
1. Texts in italic bold format are limitations quoted either directly or conceptually from claims/descriptions disclosed in the instant application.
2. Texts in regular italic format are quoted directly from cited reference or Applicant’s arguments.
3. Texts with underlining are added by the Examiner for emphasis.
4. Texts with
5. Acronym “PHOSITA” stands for “Person Having Ordinary Skill In The Art”.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claim(s) 21-23, 30 and 32-33 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 16 and 22-23 of U.S. Patent No. 12,288,300 B2. Although the claims at issue are not identical, they are not patentably distinct from each other because the claim of the instant application is either anticipated by, or the obvious variation of, the claim of U.S. Patent No. 12,288,300 B2, as shown in the table below.
Instant Application:
US 12,288,300 B2
21. A method for generating a two-dimensional (2D) image of one or more products in a physical scene, the method performed by a computing device comprising a computer hardware processor, the method comprising:
obtaining, via a communication network from another computing device, an image of the physical scene;
obtaining, via the communication network from the other computing device, position information indicative of a target position of a first product in the physical scene;
rendering a 2D image of a second product in the physical scene using the image of the physical scene, the position information, and a 3D model of the second product; and
providing, via the communication network to the other computing device, the rendered 2D image of the second product in the physical scene for display by the other computing device.
16. A method for generating a two-dimensional (2D) image of one or more products in a physical scene, the method performed by a computing device comprising a computer hardware processor, the method comprising:
obtaining a three-dimensional (3D) model of the physical scene;
generating an image of the physical scene using the 3D model of the physical scene;
determining, based on input provided by a user through a graphical user interface (GUI), position information indicative of a target position of a first product in the physical scene;
generating a plurality of 2D images of a respective plurality of products, each of the plurality of products being of a same type as the first product and being a different product from the first product, each of the plurality of 2D images including a respective image of one of the plurality of products in the physical scene using the image of the physical scene and the position information; and
displaying the plurality of 2D images of the respective plurality of products.
22
16
23
16
30
16
32
22
33
23
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 21-23, 25, 27-36, 39-41 and 43 are rejected under 35 U.S.C. 103 as being unpatentable over Canada et al. (US 10,319,150 B1).
Regarding Claim 21, Canada discloses a method (Claim 14 is a method claim) for generating a two-dimensional (2D) image of one or more products in a physical scene (Fig.2B and col.5 line 18-20: In this example, an image of a couch 222 is shown to be displayed over the view of the scene in the image 224), the method performed by a computing device comprising a computer hardware processor (Fig.13: notice the hardware processor 1302. col.2 lines 33-38: providing a virtual sticker (e.g., two-dimensional rendering) of the item, model, or other such rendering that the user can insert or otherwise "place" into the image data ( e.g., multiple still images, video, live image data, etc.) of the live camera view of the physical space), the method comprising:
obtaining, via a communication network from another computing device, an image of the physical scene (col.16 lines 25-34: In at least some embodiments, a customer can have the ability to request that others help with the design process. For example, a customer can cause a request to be sent to one or more specified people, such as friends or family, whereby that person is requested to help with the design process. That person can then obtain a view of the space, such as by logging on to a corresponding website or opening a link in an application, and can have the ability to suggest and/or select items for that space, which can then be reviewed by the customer. col.2 lines 13-18: For example, a rendering of an item can be placed within a live camera view of the physical space. A snapshot of the physical space (e.g., canvas) can be captured and the snapshot can be customized, shared, and the renderings within the snapshot can be consumed (e.g., purchased, rented, etc.));
obtaining, via the communication network from the other computing device (See Fig.11: notice Canada does not explicitly recite the position information is obtained via the communication network from the other computing device. However Canada, shown step 1110 in Fig.11, receiving selection and position data for item(s) to be positioned. Canda further discloses The environment in one embodiment is a distributed computing environment utilizing several computer systems and components that are interconnected via communication links, using one or more computer networks or direct connections, see col.22 line 49-53. Therefore it would have been obvious to a PHOSITA that the position data can be obtained through communication networks), position information indicative of a target position of a first product in the physical scene (see Fig.4B: notice the virtual lamp is placed on the actual table in the physical scene. The position information regarding the surface of the actual table is interpreted as the position information indictive of a target position of a first product in the physical scene. In order to placing the virtual lamp on the surface of the table, this position information needs to be known. Also col.11 lines 26-34: In some embodiments the mapping process can begin as soon as any portion of the data is received. As mentioned, the mapping process can depend at least in part upon the type(s) of data received. In at least some embodiments, the position and/or distance data can be used to generate a virtual model (e.g., wire frame model) of the space, and the image data can be used to provide the visual overlay (i.e., through texture mapping) of the elements in the wire model. col.18 lines 27-30: Once the user is satisfied with the selection and arrangement of the items, selection and position data can be received 1014, such as in response to the user selecting a submit or approval option);
rendering a 2D image of a second product in the physical scene using the image of the physical scene, the position information, and a 3D model of the second product (col.3 lines 10-19: Once the data for the scene is obtained, a model, mapping, or other digital representation can be generated that can enable the user and others to obtain various views of the scene, such as by using various computing devices. A user can obtain a two-dimensional view from a desired viewpoint in some embodiments, which can allow virtual stickers and other two-dimensional views of items to be virtually displayed in the environment, whether overlaying an image of the item or rendering a two-dimensional view of a three-dimensional model of the item, among other such options); and
providing, via the communication network to the other computing device, the rendered 2D image of the second product in the physical scene for display by the other computing device (col.4 lines 45-55: In accordance with various embodiments, the representation can be a virtual sticker, three-dimensional model, or other such rendering. Virtually placing items in the physical space can help the customer when browsing items on an electronic marketplace, such as to determine matching colors and patterns, as well as potential placements for various items).
Regarding Claim 22, Canada further teaches or suggests wherein the image of the physical scene and the position information are generated using a three-dimensional (3D) model of the physical scene (col.13 lines 29-51: Once a threshold number of additional images (e.g., still images, frames, etc.) are captured, then a baseline traversed during capture of additional images is determined; for example, the spatial distance between two of the camera views. By capturing the same object from different distances, positions, angles, and/or views, a multi-view stereo model may be generated that allows an estimation of a 3D geometry of the planar surface. For example, the 3D geometry may include data indicating a size and/or scale of the planar surface(s), which when evaluated along with the sensor bias value, allows for accurate sizing and rendering of a digital representation into the augmented reality display of the real-world environment in a “camera view” displayed on a device. For example, a camera view of the physical space may be displayed that includes a rendering of the physical space including the planar surface as captured by an image capture device (e.g., a live video stream), and a digital representation of an object displayed as if the object were sitting on the planar surface, with the digital representation of the object being rendered at a realistic scale in substantially real-time based on the physical dimensions of the actual object and the scale information (e.g., of the planar surface)).
Regarding Claim 23, Canada further teaches or suggests generating a 3D scene, the 3D scene including the 3D model of the second product positioned in the 3D scene in accordance with the target position; and rendering the 2D image of the second product in the physical scene using the 3D scene (col.2 lines 11-24: In particular, various embodiments provide for viewing images or renderings of items placed (virtually) within the physical space. For example, a rendering of an item can be placed within a live camera view of the physical space. A snapshot of the physical space (e.g., canvas) can be captured and the snapshot can be customized, shared, and the renderings within the snapshot can be consumed (e.g., purchased, rented, etc.). The renderings can be represented as two-dimensional images, e.g., virtual stickers, icons, or other such renderings as will be described further herein, or three-dimensional models of the items, and can be manipulated, e.g., rotated, enlarged, reduced in size, etc. The renderings can link to products offered through an electronic marketplace and those products can be consumed).
Regarding Claim 25, Canada further teaches or suggests obtaining lighting information about the physical scene; and setting lighting in the 3D scene in accordance with the lighting information (col.2 line 62-66: In at least some embodiments, additional data such as disparity data from stereoscopic images, distance data, structured light data, sensor data from a moveable camera, and the like can be utilized to provide scale and dimension data. col.17 line 14-16: The shadow effects may be dynamic and update with respect to a position and/or intensity of a physical and/or virtual light source).
Regarding Claim 27, Canada further teaches or suggests obtaining, from the other computing device via a communication network, one or more camera setting values used to capture the image of the physical scene; and rendering the 2D image of the second product in the physical scene using a virtual camera configured with the one or more camera setting values (col.8 line 30-36: If the camera is part of a device, such as a portable phone or tablet, that has motion or position sensors, for example, the motion of the device can be used to assist with scale and dimension information, as the change in the size of a representation of an object over a determinable change in distance can be used to determine the size of the object. The motion and position information are interpreted as the one or more camera setting values. Canada teaches rendering the virtual object based on the motion and position information of the camera).
Regarding Claim 28, Canada further teaches or suggests obtaining, from the other computing device via a communication network, a rotation angle of a panorama with respect to a camera position; and rendering the 2D image of the second product in the physical scene using a virtual camera configured with the rotation angle (col.5 line 20-27: The image can be a two-dimensional rendering of the produce such as a virtual sticker, or such rendering of the product. In an example, the virtual sticker can include a representation of the product as well as additional content, such as a border, a drop shadow, etc. A user can interact with the sticker. For example, the user can rotate, enlarge, reduce in size, change the location of the sticker, etc.).
Regarding Claim 29, Canada discloses wherein the image of the physical scene comprises a composite image comprising a virtual model of the first product overlaid onto the image of the physical scene (col.13 lines 2-4: For example, a planar surface selected by the user will have a digital representation of an object “placed” on it in an augmented reality view).
Regarding Claim 30, Canada discloses generating a graphical user interface (GUI) comprising a plurality of images of products in the physical scene, wherein the plurality of images includes the rendered 2D image of the second product in the physical scene; and providing the GUI for display by an application executing on the other computing device (Fig.4B and col.7 line 19-24: As shown in example 420 of FIG. 4B, in response to recognizing the physical space, products 422 related to the physical space can be displayed. The user can select products or other items of interest to be placed virtually within the physical space, as represented in example 420 of FIG. 4B. Also see Fig.5/14, Canada discloses the method can be implemented in a server/client format).
PNG
media_image1.png
510
586
media_image1.png
Greyscale
Regarding Claim 31, Canada further teaches or suggests identifying the second product using information about a user of the other computing device (col.19 lines 60-66: As mentioned, there can be various ways to determine which items to suggest for a customer space. In at least some embodiments, these can include autosuggestions based at least in part upon items purchased by the customer, items purchased by others having similar items in a similar space, items having similar features (brand, style, designer) to items purchased by the user, and the like).
Regarding Claim 32, Claim 32 is/are similar to Claim 21 except in the format of system. Canada discloses a computer hardware processor; and a non-transitory computer-readable storage medium storing instructions that, when executed by the computer hardware processor, cause the computer hardware processor to perform a method (Fig.13: notice processor 1302 and memory 1304). Therefore the same reason(s) for rejection is/are applied to Claim 21 is/are also applied to Claim 32.
Regarding Claim 33, Claim 33 is/are similar to Claim 21 except in the format of a non-transitory computer-readable storage medium. Therefore the same reason(s) for rejection is/are applied to Claim 21 is/are also applied to Claim 33.
Regarding Claim 34, Canada teaches or suggests a method for obtaining a two-dimensional (2D) image of one or more products in a physical scene, the method performed by a computing device comprising a computer hardware processor (Fig.13: notice the hardware processor 1302. col.2 lines 33-38: providing a virtual sticker (e.g., two-dimensional rendering) of the item, model, or other such rendering that the user can insert or otherwise "place" into the image data ( e.g., multiple still images, video, live image data, etc.) of the live camera view of the physical space), the method comprising:
generating an extended reality (XR) scene (col.12 line 6-16: In this example, an augmented reality (AR) application is initialized by capturing image data of a physical space. For example, a user may have a mobile device with an image capture device (e.g., a camera), and begin by taking a picture or video of the physical space desired to be “augmented” with digital representations. The image capture device can be capable of receiving a stream of image data (e.g., a live video stream) and displaying the stream of image data on a display (e.g., the mobile device screen, one or more external displays, AR goggles, display-capable eyewear such as glasses or contact lenses, a virtual reality interface, etc.). Although Canada does not explicitly use the phrase extended reality, a skilled person would have recognized that augmented, virtual, and mixed reality belong to extended reality);
generating, using the XR scene, an image of the physical scene (see Fig.4A);
determining, based on input provided by a user through the XR scene, position information indicative of a target position of a first product in the physical scene (see Fig.4B: notice the virtual lamp is placed on the actual table in the physical scene. The position information regarding the surface of the actual table is interpreted as the position information indictive of a target position of a first product in the physical scene. In order to placing the virtual lamp on the surface of the table, this position information needs to be known. col.18 lines 27-30: Once the user is satisfied with the selection and arrangement of the items, selection and position data can be received 1014, such as in response to the user selecting a submit or approval option);
generating a plurality of 2D images of a respective plurality of products, each of the plurality of products being of a same type as the first product and being a different product from the first product, each of the plurality of 2D images including an image of respective one of the plurality of products in the physical scene and being rendered using the image of the physical scene and the position information; and displaying the plurality of 2D images of the respective plurality of products (col.19 lines 18-20: As mentioned, the user can move around the renderings of the items in the virtual space, change items, potentially remove existing items from the space, etc. A PHOSITA would have recognized that changing items can be replacing existing item with a new virtual item with same type but different style. See col.7 lines 54-58: Such an approach can enable the customer to view the color, fit, style, and other aesthetic aspects of the item in the potential space, as well as determining whether the item will fit in that space, its size and spacing relative to other items, and other such aspects).
Regarding Claim 35, Canada further teaches or suggests wherein generating the plurality of 2D images of the respective plurality of products comprises generating a 2D image of a second product in the physical scene at least in part by: generating a 3D scene, the 3D scene including a 3D model of the second product positioned in the 3D scene in accordance with the target position; and rendering the 2D image of the second product in the physical scene using the 3D scene (col.18 line 63-col.19 line 20: FIG. 11 illustrates another example process 1100 that can be utilized in accordance with various embodiments. In this example, a request is received 1102 wherein a user would like to view data for items that can be purchased for a specific customer (or group of customers, etc.). ... If the user is determined to be able to receive the information, one or more views of the space can be caused to be provided 1104 to the user as discussed above. Potential items for the space can be determined 1106, such as by analyzing customer data, analyzing a wish list or registry, determining items of a specified category or style, etc. ... Models, images, and/or renderings of various items then can be provided 1108 to the user device, in order to enable the user to obtain scale-appropriate views of the items in various locations in the space. As mentioned, the user can move around the renderings of the items in the virtual space, change items, potentially remove existing items from the space, etc.).
Regarding Claim 36, Canada teaches or suggests obtaining lighting information indicating one or more light sources in the physical scene, wherein generating the 2D image of the second product in the physical scene comprises setting lighting in the 3D scene in accordance with the lighting information (col.17 lines 12-17: A drop shadow or other shadow effect may be applied to the border and/or representation of the product. The shadow effects may be dynamic and update with respect to a position and/or intensity of a physical and/or virtual light source. Various algorithms may be implemented to generate such effects).
Regarding Claim 39, Claim 39 is/are similar to Claim 34 except in the format of system. Canada discloses a system comprises: a camera (Fig.1 and col.4 line 31-34: for example, a real-time or near-real-time rendering of the image data being captured by the camera 114, which in this example would be of the physical space including the end tables); a computer hardware processor; and a non-transitory computer-readable storage medium storing instructions that, when executed by the computer hardware processor (Fig.13: notice processor 1302 and memory 1304), cause the computer hardware processor to perform instructions stored in the processor. Therefore the same reason(s) for rejection is/are applied to Claim 34 is/are also applied to Claim 39.
Regarding Claim 40-41, Claim 40-41 is/are similar to Claim 35-36 except in the format of system. Therefore the same reason(s) for rejection is/are applied to Claim 35-36 is/are also applied to Claim 40-41.
Regarding Claim 43, Claim 43 is/are similar to Claim 34 except in the format of a non-transitory computer-readable storage medium. Therefore the same reason(s) for rejection is/are applied to Claim 34 is/are also applied to Claim 43.
Claims 24, 26, 38 and 42 are rejected under 35 U.S.C. 103 as being unpatentable over Canada et al. (US 10,319,150 B1) as applied to Claim 23, 34 and 39 above, and further in view of Tomite et al. (US 2008/0024523 A1).
Regarding Claim 24, Canada fails to disclose projecting the image of the physical scene onto a sphere to obtain a spherical mapping of the image of the physical scene; and applying the spherical mapping to the 3D scene to set a background environment in the 3D scene to the image of the physical scene.
However Tomite, in the same field of endeavor, discloses
projecting the image of the physical scene onto a sphere to obtain a spherical mapping of the image of the physical scene ([0033]: In FIG. 2, a head mounted display (hereinafter, referred to as HMD) 101, carried by a user 201, includes a video camera capable of obtaining position and orientation data. The user 201 can observe a real space through a video image (real image) 205 captured by the video camera. In FIG. 2, 5. Each captured image 205 is an image of the real space captured by the video camera. The celestial sphere 206 is a hemisphere having an infinite radius and can be regarded as a screen on which the captured image 205 can be projected); and
PNG
media_image2.png
314
325
media_image2.png
Greyscale
applying the spherical mapping to the 3D scene to set a background environment in the 3D scene to the image of the physical scene ([0034]-[0035]: In the present exemplary embodiment, the information processing apparatus 100 estimates light source information of the real space based on the generated celestial sphere image. The information processing apparatus 100 generates a virtual object 204 having natural brightness and shadow based on the estimated light source information. The information processing apparatus 100 combines the virtual object 204 having the brightness and shadow being set as described above with a real image of a desk 203 (i.e., a real object) captured by the video camera of the user 201 ).
Tomite and Canada both are regarding generating Augmented/Mixed reality application by superimposing a virtual image to a real scene. Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Tomite into that of Canada and to include the limitation of projecting the image of the physical scene onto a sphere to obtain a spherical mapping of the image of the physical scene; and applying the spherical mapping to the 3D scene to set a background environment in the 3D scene to the image of the physical scene in order to correctly recognize the information and state of the real space and accurately input the information of the real space to the virtual space as suggested by Tomite ([0009]).
Regarding Claim 26, Canada teaches or suggests obtaining an indication of a plane in the physical scene (col.12 line 53-col.13 line 4: After determining a sensor bias value, a user is prompted to point the image capture device at a surface (e.g., a planar surface such as a table, a wall, a painting on a wall, a book on a table, a container, etc.). A GUI element such as a “target” or “focus area” comprising a subset of a display screen on which the stream of image data being captured from the real-world environment is being displayed is presented to a user in order to allow the user to “place” the desired planar surface in the target area for processing. A user captures image data of the physical space and some or all potential planar surfaces are automatically identified and visually indicated in a GUI of a device. A user captures image data of the physical space and provides an indication of a selection of one or more areas of the image data that the user would like to have evaluated for the presence of a suitable planar surface, or identifying a desired planar surface, etc. For example, a planar surface selected by the user will have a digital representation of an object “placed” on it in an augmented reality view).
But Canada fails to explicitly disclose using the indication of the plane in the physical scene to generate lighting effects in the 3D scene.
However Tomite shows in Fig.2, placing a virtual object 204 on a real image of a desk 203 and The information processing apparatus 100 combines the virtual object 204 having the brightness and shadow being set as described above with a real image of a desk 203 (i.e., a real object) captured by the video camera of the user 201 ([0035]).
PNG
media_image2.png
314
325
media_image2.png
Greyscale
Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Tomite into that of Canada and to include the limitation of obtaining an indication of a plane in the physical scene; and using the indication of the plane in the physical scene to generate lighting effects in the 3D scene (the desk top surface is interpreted as the plane and the brightness and shadow are interpreted as the lighting effects) in order to correctly recognize the information and state of the real space and accurately input the information of the real space to the virtual space as suggested by Tomite ([0009]).
Regarding Claim 37, Canada disclose rendering virtual objects in real scene based on condition of light source (col.17 to line 14-16: The shadow effects may be dynamic and update with respect to a position and/or intensity of a physical and/or virtual light source). But Canada does not explicitly disclose receiving, through an XR interface, user input identifying the one or more light sources in the physical scene; and determining one or more locations of the one or more light sources in the physical scene based on the user input.
However Tomite, in the same field of endeavor, discloses allowing a user to change light source information ([0056]: a user can edit the light source information during an operation of the system. [0061]: Although the light source of the present exemplary embodiment is a point light source, it may be useful to deal with other kinds of light source or edit the light source information based on a user's instruction. [0064]: Furthermore, the user 201 can change the light source information registered in the virtual space database 108 by changing the light source information managed by the light source information managing unit 107). Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Tomite into that of Canada and to include the limitation of receiving, through an XR interface, user input identifying the one or more light sources in the physical scene; and determining one or more locations of the one or more light sources in the physical scene based on the user input in order to allow to change virtual light source as users wish.
Regarding Claim 38, Canada teaches or suggests obtaining an indication of a plane in the physical scene (col.12 line 53-col.13 line 4: After determining a sensor bias value, a user is prompted to point the image capture device at a surface (e.g., a planar surface such as a table, a wall, a painting on a wall, a book on a table, a container, etc.). A GUI element such as a “target” or “focus area” comprising a subset of a display screen on which the stream of image data being captured from the real-world environment is being displayed is presented to a user in order to allow the user to “place” the desired planar surface in the target area for processing. A user captures image data of the physical space and some or all potential planar surfaces are automatically identified and visually indicated in a GUI of a device. A user captures image data of the physical space and provides an indication of a selection of one or more areas of the image data that the user would like to have evaluated for the presence of a suitable planar surface, or identifying a desired planar surface, etc. For example, a planar surface selected by the user will have a digital representation of an object “placed” on it in an augmented reality view).
But Canada fails to explicitly disclose wherein generating the 2D image of the second product in the physical scene comprises using the indication of the plane in the physical scene to generate lighting effects in the 3D scene.
However Tomite shows in Fig.2, placing a virtual object 204 on a real image of a desk 203 and The information processing apparatus 100 combines the virtual object 204 having the brightness and shadow being set as described above with a real image of a desk 203 (i.e., a real object) captured by the video camera of the user 201 ([0035]).
PNG
media_image2.png
314
325
media_image2.png
Greyscale
Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Tomite into that of Canada and to include the limitation of obtaining an indication of a plane in the physical scene, wherein generating the 2D image of the second product in the physical scene comprises using the indication of the plane in the physical scene to generate lighting effects in the 3D scene (the desk top surface is interpreted as the plane and the brightness and shadow are interpreted as the lighting effects) in order to correctly recognize the information and state of the real space and accurately input the information of the real space to the virtual space as suggested by Tomite ([0009]).
Regarding Claim 42, Claim 42 is/are similar to Claim 38 except in the format of system. Therefore the same reason(s) for rejection is/are applied to Claim 38 is/are also applied to Claim 42.
wherein generating the 2D image of the second product in the physical scene comprises using the indication of the plane in the physical scene to generate lighting effects in the 3D scene.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YINGCHUN HE whose telephone number is (571)270-7218. The examiner can normally be reached M-F 8:00-5:00 MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao M Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YINGCHUN HE/Primary Examiner, Art Unit 2613
Prosecution Insights