DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The following is a Final Office Action in response to communications received on 3/23/2026. Claims 1-24 are currently pending and have been examined. Claims 1, 8, 9, 16, 17, and 24 have been amended.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5, 8-13, 16-21, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Ayush (US 20210133850) in view of Faranda (US 20120120214).
Regarding claims 1, 9, and 17, Ayush discloses:
A method, comprising: (claim 1)
An electronic device, comprising: one or more processors; memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: (Figure 1 Machine learning prediction system communicating with user device) (claim 9)
A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to: [0042] The example data storage unit 124 can include one or more non-transitory computer-readable media and can be either included in the machine learning prediction system 120 or operatively coupled to the machine learning prediction system 120. The data storage unit 124 can include on-board flash memory and/or one or more removable memory cards or removable flash memory. (claim 17)
at an electronic device in communication with a display (user device and display 113) and one or more input devices (camera 117): (Figure 1) presenting, via the display, a representation of one or more virtual objects in a three-dimensional environment; (shown in Figure 3 augmented product C) [0075] FIG. 7 shows several example potential viewpoints 700, 702, 704 of an AR environment, in accordance with an embodiment of the present disclosure. Each of the viewpoints 700, 702, 704 represents a different perspective of a real-world environment from which to visualize one or more virtual products. In this example, the real-world environment includes a room with a sofa and a chair. Further, the viewpoints 700, 702, 704, depict a virtual table placed in the center of the room. The virtual objects are thus virtually placed in the camera image of the physical objects to produce the AR viewpoint. Each of the viewpoints 700, 702, 704 thus allows the user to visualize the room, with the physical objects (e.g., sofa and chair) and the virtual objects (e.g., table), from different perspectives or viewpoints.
in accordance with detecting a physical object that corresponds to a virtual object of the one or more virtual objects, (shown in Figure 2 real world objects A and B) selecting the virtual object for comparison with the physical object; [0018]The embedded image of the user-selected virtual product is inserted into the camera image of the real-world object, to provide the AR viewpoint. A candidate product that is complementary to the user-selected virtual product is then machine-selected (i.e., machine-predicted) to provide a product bundle that includes both the virtual product and the candidate product. [0058]The table C can be selected by the customer or recommended to the customer by comparing the sofa A and/or the bookshelf B to an inventory of products for sale, such as an inventory of tables. And see [0061] and [0063]
presenting, via the display, a representation of the selected virtual object at a comparison position defined relative to a position of the physical object. [0090] FIG. 9 shows an example bounding box 900 with coordinate pairs (x1, y1) and (x2, y2), in accordance with an embodiment. For example, in the above-mentioned camera frame of the viewpoint 704, there will be different bounding boxes corresponding to different objects. FIG. 10A shows the example viewpoint 704 including the physical objects (e.g., sofa and chair) and excluding the virtual objects (e.g., table). FIG. 10A further shows bounding boxes 1000, 1002 representing the physical objects. FIG. 10B shows the example viewpoint 704 including the physical objects (e.g., sofa and chair) and the virtual objects (e.g., table). FIG. 10B further shows bounding box 1004 representing the virtual objects. [0091] In some embodiments, the bounding box for each identified object is generated using a Region Proposal Network (RPN) that shares full-image convolutional features is used with the detection network (for example, R-CNN), thus enabling nearly cost-free region proposals as represented by the bounding boxes. An RPN is a fully convolutional network that simultaneously predicts object bounds and so-called objectness scores at each position in the viewpoint. And shown in Figure 10B
While Ayush discloses the simulation of real world products with virtual products for the purposes of comparing items, the reference does not expressly disclose:
while presenting the representation of the selected virtual object at the comparison position detecting a reorientation of the physical object; and
in response to detecting the reorientation of the physical object, reorienting, via the display, the representation of the selected virtual object in accordance with the reorientation of the physical object.
However Faranda teaches:
while presenting the representation of the selected virtual object at the comparison position [0058] Additionally, in such embodiments, comparison data among the plurality of products may be provides such as by the virtual menu options, detecting a reorientation of the physical object; and shown in Figure 3 display and [0038] Once the product 302a has been identified, the product demonstrator 102 can determine product-specific information, as well as a product-specific alteration to make to the real-time video image. As an example, in addition to rendering a virtual product 302b and a virtual user 304b, the product demonstrator 102 may additionally alter the real-time video image to provide a virtual product alteration 302c. More specifically, in the embodiment of FIG. 3, the product 302a is packaged and, as such, the alteration may include presenting an image of the product outside of the packaging. Additional product information may also be provided via selection of a more detail option 308.
in response to detecting the reorientation of the physical object, reorienting, via the display, the representation of the selected virtual object in accordance with the reorientation of the physical object. [0045] FIG. 6 depicts an interface from the product demonstrator 102, illustrating product information, according to embodiments shown and described herein. As shown in FIG. 6, the user 304a has rotated the product 302a to correspond with the product information virtual menu option 402b (and/or made another physical gesture). Additionally, the user 304a has performed a triggering action to select the product information virtual menu option 402b. In response, the product demonstrator 102 can alter the real-time video image to provide the virtual product 602, as well as one or more indicators 602a-602f for providing information related to the virtual product 602. More specifically, in the example from FIG. 6, the Oral B Electric Toothbrush, Model PC5000 includes an electric toothbrush body 602a, a toothbrush head 602b, a toothbrush base and charger 602c, a smart guide 602d, a travel case 602e, and a replacement toothbrush head 602f. Additionally, a "more details" option 604 may be included for providing additional product information. And see details of orientation at [0055]
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the presentation of the real world products with virtual products for the purposes of comparing items in Ayush to include while presenting the representation of the selected virtual object at the comparison position detecting a reorientation of the physical object; and in response to detecting the reorientation of the physical object, reorienting, via the display, the representation of the selected virtual object in accordance with the reorientation of the physical object, as taught in Faranda, in order to provide the consumer with more information on the product (see paragraph 002).
Regarding claims 2, 10, and 18 Ayush in view of Faranda teaches the limitations set forth above. While Ayush discloses the simulation of real world products with virtual products for the purposes of comparing items, the reference does not expressly disclose:
wherein the representation of the one or more virtual objects comprises one or more object cards
However Faranda teaches:
wherein the representation of the one or more virtual objects comprises one or more object cards (see Price and Coupons display in the interface of Figure 5 and [0043] As shown, in response to a physical gesture, such as holding the product 302a in a predetermined orientation for a predetermined amount of time (or otherwise performing a triggering action), information related to price and coupons for the product 302a may be provided via a menu overlay 502. The menu overlay 502 may include price information, as well as a "find coupons" option 504, and an "historical price data" option 506. By selecting the find coupons option 504, information regarding available coupons and/or discounts for the product 302a may be provided. Available coupons may be stored locally on the product demonstrator 102 and/or may be stored remotely and accessible via a wide area network and/or local area network. In some embodiments, the product demonstrator 102 may be configured to print (or be coupled to a printing device, such as a printer) a rebate, a coupon, product use information, product feature information, and/or other information.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the presentation of the real world products with virtual products for the purposes of comparing items in Ayush to include wherein the representation of the one or more virtual objects comprises one or more object cards, as taught in Faranda, in order to provide the consumer with more information on the product (see paragraph 002).
Regarding claim 3, Ayush in view of Faranda teaches the limitations set forth above and Ayush further discloses:
wherein the representation of the one or more virtual objects comprises one or more images of the one or more virtual objects. (see Figures 12 and 13 coffee table, lamp and vase augmented reality items in viewpoint with physical objects "sofa")
Regarding claim 4, Ayush in view of Faranda teaches the limitations set forth above and Ayush further discloses:
wherein selecting the virtual object for comparison with the physical object is performed further in accordance with detecting movement of the physical object to less than a first threshold distance from a representation of the virtual object. [0094] Referring again to FIG. 6B, identifying 610 the one or more objects in the viewpoint of the AR environment further includes identifying 616 a pose, or orientation, of each real-world and virtual objects identified in the selected viewpoint (e.g., the viewpoint 704). It is important to note that the poses of the real-world objects and of the virtual objects in the viewpoint are identified in contrast to some existing techniques where only the poses of real-world objects in the viewpoint are identified using the disclosed techniques. To create a product bundle recommendation, the pose of the identified objects (real-world and virtual) is identified in the selected viewpoint. [0095] In some embodiments, correlation filters (CF) can be used in the bounding boxes of the identified objects to figure out their pose in the viewpoint with respect to camera coordinates. CF-based training uses images containing a class, such as a class of “table” representing table products. The class includes a predetermined number of images of tables on a monochromatic background where each table in the class is represented by images of the table in multiple different orientations or poses. In this manner, a virtual representation of the table can be displayed in a pose that closely corresponds to the identified pose of one or more of the real-world and virtual objects. Correlation filters can be used to control the shape of the cross-correlation output between the image and the filter by minimizing the average Mean Square Error (MSE) between the cross-correlation output and the ideal desired correlation output for an authentic (or impostor) input image. By explicitly controlling the shape of the entire correlation output, unlike traditional classifiers which only control the output value at the target location, CFs achieve more accurate local estimation.
Regarding claim 5, Ayush in view of Faranda teaches the limitations set forth above and Ayush further discloses:
detecting movement of the physical object to less than a second threshold distance from a particular representation of the one or more virtual objects, but greater than a first threshold distance, the second threshold distance greater than the first threshold distance; and in accordance with a determination that the physical object is less than the second threshold distance but greater than the first threshold distance, relocating the particular representation of the one or more virtual objects closer to the physical object. [0070] The user 101 then positions the virtual product within the camera image. After the user 101 selects a product to be virtually placed into the real-world environment depicted on the user device 110, the user 101 utilizes the AR application 115 along with the camera 117 to move the virtual product around within the camera image. For example, after the virtual product is inserted in the camera image, the user 101 can move the user device 110 to position the virtual product in the desired location, orientation, and scale in the camera image. The desired location, for example, corresponds to the location, orientation, and scale in the user's surroundings where the user 101 wishes to place an actual product corresponding to the virtual product. If, for example, the camera image includes a sofa and chair in the user's living room, and the virtual product is of a table that the user 101 is interested in purchasing, the user 101 may move the user device 110 (and the associated camera 117) to position the virtual table in the center of the room as desired. [0071] In certain embodiments, in addition to moving the user device 110 to position the virtual product, the user may move the virtual product to a specific location within the camera image, such as on the top, bottom, left, right, or center of the camera image. For example, the user 101 may drag the virtual product in the user device display 113 and reposition the virtual product within the camera image. If the virtual product is an image of a table, for example, the user may drag the virtual product around in the camera image to a desired location. [0072] For example, user input can shrink or enlarge the virtual product relative to real objects in the camera image.
Regarding claims 8, 16, and 24, Ayush discloses the limitations set forth above.
Ayush further discloses:
and in response to detecting the reorientation of the physical object and reorienting the representation of the selected virtual object: (see Figure 7 showing the different viewpoints of the real world objects in conjunction with the AR object) [0094] Referring again to FIG. 6B, identifying 610 the one or more objects in the viewpoint of the AR environment further includes identifying 616 a pose, or orientation, of each real-world and virtual objects identified in the selected viewpoint (e.g., the viewpoint 704). It is important to note that the poses of the real-world objects and of the virtual objects in the viewpoint are identified in contrast to some existing techniques where only the poses of real-world objects in the viewpoint are identified using the disclosed techniques. To create a product bundle recommendation, the pose of the identified objects (real-world and virtual) is identified in the selected viewpoint.
While Ayush discloses the simulation of real world products with virtual products for the purposes of comparing items, the reference does not expressly disclose:
selectively presenting, via the display, one or more textual indicators associated with one or both of the physical object and the representation of the selected virtual object in accordance with a particular surface of the physical object or the representation of the selected virtual object being exposed during the reorientation or in accordance with a particular feature area on or above the physical object or the representation of the selected virtual object being exposed during the reorientation, each textual indicator presenting information for comparing the physical object and the selected virtual object.
However Faranda teaches:
selectively presenting, via the display, one or more textual indicators associated with one or both of the physical object and the representation of the selected virtual object in accordance with a particular surface of the physical object or the representation of the selected virtual object being exposed during the reorientation or in accordance with a particular feature area on or above the physical object or the representation of the selected virtual object being exposed during the reorientation, each textual indicator presenting information for comparing the physical object and the selected virtual object. [0045] FIG. 6 depicts an interface from the product demonstrator 102, illustrating product information, according to embodiments shown and described herein. As shown in FIG. 6, the user 304a has rotated the product 302a to correspond with the product information virtual menu option 402b (and/or made another physical gesture). Additionally, the user 304a has performed a triggering action to select the product information virtual menu option 402b. In response, the product demonstrator 102 can alter the real-time video image to provide the virtual product 602, as well as one or more indicators 602a-602f for providing information related to the virtual product 602. More specifically, in the example from FIG. 6, the Oral B Electric Toothbrush, Model PC5000 includes an electric toothbrush body 602a, a toothbrush head 602b, a toothbrush base and charger 602c, a smart guide 602d, a travel case 602e, and a replacement toothbrush head 602f. Additionally, a "more details" option 604 may be included for providing additional product information. And see details of orientation at [0055] and see at least Figure 4
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the comparison of the products in Ayush to include further selectively presenting, via the display, one or more textual indicators associated with one or both of the physical object and the representation of the selected virtual object in accordance with a particular surface of the physical object or the representation of the selected virtual object being exposed during the reorientation or in accordance with a particular feature area on or above the physical object or the representation of the selected virtual object being exposed during the reorientation, each textual indicator presenting information for comparing the physical object and the selected virtual object, as taught in Faranda, in order to in order to provide the consumer with more information on the product (see paragraph 002).
The examiner notes the limitations “in response to detecting the reorientation of the physical object and reorienting the representation of the selected virtual object:
selectively presenting, via the display, […]” is recited as conditional language and not positively recited. The limitation has been addressed with prior art to promote compact prosecution, however the limitation is given little patentable weight.
Regarding claim 11, Ayush in view of Faranda teaches the limitations set forth above and Ayush further discloses:
wherein the representation of the one or more virtual objects comprises one or more images of the one or more virtual objects. (see Figures 12 and 13 coffee table, lamp and vase augmented reality items in viewpoint with physical objects "sofa")
Regarding claim 12, Ayush in view of Faranda teaches the limitations set forth above and Ayush further discloses:
wherein selecting the virtual object for comparison with the physical object is performed further in accordance with detecting movement of the physical object to less than a first threshold distance from a representation of the virtual object. [0094] Referring again to FIG. 6B, identifying 610 the one or more objects in the viewpoint of the AR environment further includes identifying 616 a pose, or orientation, of each real-world and virtual objects identified in the selected viewpoint (e.g., the viewpoint 704). It is important to note that the poses of the real-world objects and of the virtual objects in the viewpoint are identified in contrast to some existing techniques where only the poses of real-world objects in the viewpoint are identified using the disclosed techniques. To create a product bundle recommendation, the pose of the identified objects (real-world and virtual) is identified in the selected viewpoint.
[0095] In some embodiments, correlation filters (CF) can be used in the bounding boxes of the identified objects to figure out their pose in the viewpoint with respect to camera coordinates. CF-based training uses images containing a class, such as a class of “table” representing table products. The class includes a predetermined number of images of tables on a monochromatic background where each table in the class is represented by images of the table in multiple different orientations or poses. In this manner, a virtual representation of the table can be displayed in a pose that closely corresponds to the identified pose of one or more of the real-world and virtual objects. Correlation filters can be used to control the shape of the cross-correlation output between the image and the filter by minimizing the average Mean Square Error (MSE) between the cross-correlation output and the ideal desired correlation output for an authentic (or impostor) input image. By explicitly controlling the shape of the entire correlation output, unlike traditional classifiers which only control the output value at the target location, CFs achieve more accurate local estimation.
Regarding claim 13, Ayush in view of Faranda teaches the limitations set forth above and Ayush further discloses:
detecting movement of the physical object to less than a second threshold distance from a particular representation of the one or more virtual objects, but greater than a first threshold distance, the second threshold distance greater than the first threshold distance; and in accordance with a determination that the physical object is less than the second threshold distance but greater than the first threshold distance, relocating the particular representation of the one or more virtual objects closer to the physical object. [0070] The user 101 then positions the virtual product within the camera image. After the user 101 selects a product to be virtually placed into the real-world environment depicted on the user device 110, the user 101 utilizes the AR application 115 along with the camera 117 to move the virtual product around within the camera image. For example, after the virtual product is inserted in the camera image, the user 101 can move the user device 110 to position the virtual product in the desired location, orientation, and scale in the camera image. The desired location, for example, corresponds to the location, orientation, and scale in the user's surroundings where the user 101 wishes to place an actual product corresponding to the virtual product. If, for example, the camera image includes a sofa and chair in the user's living room, and the virtual product is of a table that the user 101 is interested in purchasing, the user 101 may move the user device 110 (and the associated camera 117) to position the virtual table in the center of the room as desired. [0071] In certain embodiments, in addition to moving the user device 110 to position the virtual product, the user may move the virtual product to a specific location within the camera image, such as on the top, bottom, left, right, or center of the camera image. For example, the user 101 may drag the virtual product in the user device display 113 and reposition the virtual product within the camera image. If the virtual product is an image of a table, for example, the user may drag the virtual product around in the camera image to a desired location. [0072] For example, user input can shrink or enlarge the virtual product relative to real objects in the camera image.
Regarding claim 19, Ayush in view of Faranda teaches the limitations set forth above and Ayush further discloses:
wherein the representation of the one or more virtual objects comprises one or more images of the one or more virtual objects. (see Figures 12 and 13 coffee table, lamp and vase augmented reality items in viewpoint with physical objects "sofa")
Regarding claim 20, Ayush in view of Faranda teaches the limitations set forth above and Ayush further discloses:
wherein selecting the virtual object for comparison with the physical object is performed further in accordance with detecting movement of the physical object to less than a first threshold distance from a representation of the virtual object. [0094] Referring again to FIG. 6B, identifying 610 the one or more objects in the viewpoint of the AR environment further includes identifying 616 a pose, or orientation, of each real-world and virtual objects identified in the selected viewpoint (e.g., the viewpoint 704). It is important to note that the poses of the real-world objects and of the virtual objects in the viewpoint are identified in contrast to some existing techniques where only the poses of real-world objects in the viewpoint are identified using the disclosed techniques. To create a product bundle recommendation, the pose of the identified objects (real-world and virtual) is identified in the selected viewpoint.
[0095] In some embodiments, correlation filters (CF) can be used in the bounding boxes of the identified objects to figure out their pose in the viewpoint with respect to camera coordinates. CF-based training uses images containing a class, such as a class of “table” representing table products. The class includes a predetermined number of images of tables on a monochromatic background where each table in the class is represented by images of the table in multiple different orientations or poses. In this manner, a virtual representation of the table can be displayed in a pose that closely corresponds to the identified pose of one or more of the real-world and virtual objects. Correlation filters can be used to control the shape of the cross-correlation output between the image and the filter by minimizing the average Mean Square Error (MSE) between the cross-correlation output and the ideal desired correlation output for an authentic (or impostor) input image. By explicitly controlling the shape of the entire correlation output, unlike traditional classifiers which only control the output value at the target location, CFs achieve more accurate local estimation.
Regarding claim 21, Ayush in view of Faranda teaches the limitations set forth above and Ayush further discloses:
detecting movement of the physical object to less than a second threshold distance from a particular representation of the one or more virtual objects, but greater than a first threshold distance, the second threshold distance greater than the first threshold distance; and in accordance with a determination that the physical object is less than the second threshold distance but greater than the first threshold distance, relocating the particular representation of the one or more virtual objects closer to the physical object. [0070] The user 101 then positions the virtual product within the camera image. After the user 101 selects a product to be virtually placed into the real-world environment depicted on the user device 110, the user 101 utilizes the AR application 115 along with the camera 117 to move the virtual product around within the camera image. For example, after the virtual product is inserted in the camera image, the user 101 can move the user device 110 to position the virtual product in the desired location, orientation, and scale in the camera image. The desired location, for example, corresponds to the location, orientation, and scale in the user's surroundings where the user 101 wishes to place an actual product corresponding to the virtual product. If, for example, the camera image includes a sofa and chair in the user's living room, and the virtual product is of a table that the user 101 is interested in purchasing, the user 101 may move the user device 110 (and the associated camera 117) to position the virtual table in the center of the room as desired. [0071] In certain embodiments, in addition to moving the user device 110 to position the virtual product, the user may move the virtual product to a specific location within the camera image, such as on the top, bottom, left, right, or center of the camera image. For example, the user 101 may drag the virtual product in the user device display 113 and reposition the virtual product within the camera image. If the virtual product is an image of a table, for example, the user may drag the virtual product around in the camera image to a desired location. [0072] For example, user input can shrink or enlarge the virtual product relative to real objects in the camera image.
Claims 6, 14, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Ayush (US 20210133850) in view of Faranda (US 20120120214) in further view of Panse (US 20210398316)
Regarding claim 6, Ayush in view of Faranda teaches the limitations set forth above. While Ayush discloses the simulation of real world products with virtual products for the purposes of comparing items, the combination of Ayush and Faranda does not expressly disclose:
wherein the relocation of the particular representation of the one or more virtual objects closer to the physical object simulates magnetic attraction
However Panse teaches:
wherein the relocation of the particular representation of the one or more virtual objects closer to the physical object simulates magnetic attraction [0134] In a second embodiment of stage S136, the augmented reality assessment includes a virtual assessment of a positioning of additional virtual objects. In practice, controller 60 may snap one virtual object next to another virtual object, or may keep one virtual object away from another virtual content so as not to interfere. [0136] Once the virtual object is positioned within the display, controller 60 loops through stages S134-S138 to constantly controlling the position and visibility based on any changes to the physical world and/or movements of physical objects. More particularly, when a virtual object interacts with a physical object, a few scenarios may occur.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the comparison of the products in Ayush in view of Faranda to include wherein the relocation of the particular representation of the one or more virtual objects closer to the physical object simulates magnetic attraction, as taught in Panse, in order to avoid inference between multiple items presented for display (paragraph 0134).
Regarding claim 14, Ayush in view of Faranda teaches the limitations set forth above. While Ayush discloses the simulation of real world products with virtual products for the purposes of comparing items, the combination of Ayush and Faranda does not expressly disclose:
wherein the relocation of the particular representation of the one or more virtual objects closer to the physical object simulates magnetic attraction
However Panse teaches:
wherein the relocation of the particular representation of the one or more virtual objects closer to the physical object simulates magnetic attraction [0134] In a second embodiment of stage S136, the augmented reality assessment includes a virtual assessment of a positioning of additional virtual objects. In practice, controller 60 may snap one virtual object next to another virtual object, or may keep one virtual object away from another virtual content so as not to interfere. [0136] Once the virtual object is positioned within the display, controller 60 loops through stages S134-S138 to constantly controlling the position and visibility based on any changes to the physical world and/or movements of physical objects. More particularly, when a virtual object interacts with a physical object, a few scenarios may occur.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the comparison of the products in Ayush in view of Faranda to include wherein the relocation of the particular representation of the one or more virtual objects closer to the physical object simulates magnetic attraction, as taught in Panse, in order to avoid inference between multiple items presented for display (paragraph 0134).
Regarding claim 22, Ayush in view of Faranda teaches the limitations set forth above. While Ayush discloses the simulation of real world products with virtual products for the purposes of comparing items, the combination of Ayush and Faranda does not expressly disclose:
wherein the relocation of the particular representation of the one or more virtual objects closer to the physical object simulates magnetic attraction
However Panse teaches:
wherein the relocation of the particular representation of the one or more virtual objects closer to the physical object simulates magnetic attraction [0134] In a second embodiment of stage S136, the augmented reality assessment includes a virtual assessment of a positioning of additional virtual objects. In practice, controller 60 may snap one virtual object next to another virtual object, or may keep one virtual object away from another virtual content so as not to interfere. [0136] Once the virtual object is positioned within the display, controller 60 loops through stages S134-S138 to constantly controlling the position and visibility based on any changes to the physical world and/or movements of physical objects. More particularly, when a virtual object interacts with a physical object, a few scenarios may occur.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the comparison of the products in Ayush in view of Faranda to include wherein the relocation of the particular representation of the one or more virtual objects closer to the physical object simulates magnetic attraction, as taught in Panse, in order to avoid inference between multiple items presented for display (paragraph 0134).
Claims 7, 15, and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Ayush (US 20210133850) in view of Faranda (US 20120120214) in further view of BRONICKIUS (US 20220374970)
Regarding claim 7, Ayush in view of Faranda teaches the limitations set forth above. While Ayush discloses the simulation of real world products with virtual products for the purposes of comparing items, the combination of Ayush in view of Faranada does not expressly disclose:
further comprising presenting one or more indicators associated with one or both of the physical object and the selected virtual object, each visual indicator presenting information for comparing the physical object and the selected virtual object.
However Bronickius teaches:
further comprising presenting one or more indicators associated with one or both of the physical object and the selected virtual object, each visual indicator presenting information for comparing the physical object and the selected virtual object. [0455] FIG. 27A illustrates an example hybrid visualization interface 2700 that may be displayed to a user, consistent with the present disclosure. Hybrid visualization interface 2700 may be an interface configured to display a combination of product images as well as synthetic image representations of the products. And see [0458-459] shown in Fig. 27B the comparison of the milk on Aisle 16 shelf 3 and an alternative for $4.29
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the comparison of the products in Ayush in view of Faranda to include further comprising presenting one or more indicators associated with one or both of the physical object and the selected virtual object, each visual indicator presenting information for comparing the physical object and the selected virtual object, as taught in Bronickius, in order to increase productivity, among other potential benefits, there is a technological need to provide a dynamic solution that will automatically monitor retail spaces. (paragraph 003).
Regarding claim 15, Ayush in view of Faranda teaches the limitations set forth above. While Ayush discloses the simulation of real world products with virtual products for the purposes of comparing items, the combination of Ayush in view of Faranada does not expressly disclose:
further comprising presenting one or more indicators associated with one or both of the physical object and the selected virtual object, each visual indicator presenting information for comparing the physical object and the selected virtual object.
However Bronickius teaches:
further comprising presenting one or more indicators associated with one or both of the physical object and the selected virtual object, each visual indicator presenting information for comparing the physical object and the selected virtual object. [0455] FIG. 27A illustrates an example hybrid visualization interface 2700 that may be displayed to a user, consistent with the present disclosure. Hybrid visualization interface 2700 may be an interface configured to display a combination of product images as well as synthetic image representations of the products. And see [0458-459] shown in Fig. 27B the comparison of the milk on Aisle 16 shelf 3 and an alternative for $4.29
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the comparison of the products in Ayush in view of Faranda to include further comprising presenting one or more indicators associated with one or both of the physical object and the selected virtual object, each visual indicator presenting information for comparing the physical object and the selected virtual object, as taught in Bronickius, in order to increase productivity, among other potential benefits, there is a technological need to provide a dynamic solution that will automatically monitor retail spaces. (paragraph 003).
Regarding claim 23, Ayush in view of Faranda teaches the limitations set forth above. While Ayush discloses the simulation of real world products with virtual products for the purposes of comparing items, the combination of Ayush in view of Faranada does not expressly disclose:
further comprising presenting one or more indicators associated with one or both of the physical object and the selected virtual object, each visual indicator presenting information for comparing the physical object and the selected virtual object.
However Bronickius teaches:
further comprising presenting one or more indicators associated with one or both of the physical object and the selected virtual object, each visual indicator presenting information for comparing the physical object and the selected virtual object. [0455] FIG. 27A illustrates an example hybrid visualization interface 2700 that may be displayed to a user, consistent with the present disclosure. Hybrid visualization interface 2700 may be an interface configured to display a combination of product images as well as synthetic image representations of the products. And see [0458-459] shown in Fig. 27B the comparison of the milk on Aisle 16 shelf 3 and an alternative for $4.29
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the comparison of the products in Ayush in view of Faranda to include further comprising presenting one or more indicators associated with one or both of the physical object and the selected virtual object, each visual indicator presenting information for comparing the physical object and the selected virtual object, as taught in Bronickius, in order to increase productivity, among other potential benefits, there is a technological need to provide a dynamic solution that will automatically monitor retail spaces. (paragraph 003).
Response to Arguments
Subject Matter Eligibility
With respect to the remarks filed 3/23/2026 directed to 35 USC 101, the examiner finds the remarks “claim 1 integrates the judicial exception into a practical application. As described above, the claim recites detecting orientation or reorientation of the physical object in the physical world (e.g., using one or more input devices, such as orientation sensors or image sensors described at paragraphs [0039]-[0046]), and updating display of the corresponding virtual objects to correspond to the detected orientation. These recitations are not merely "data collection" or "display" operations used to apply the judicial exception to a "field of use" of a three- dimensional environment, but rather require technical processes to monitor the movement (position and orientation) of a real-world object, and update of the display of a virtual object in a comparison position defined relative to the position of the real-world object in response to the reorientation of the physical object. These recitations represent a technical improvement by synchronizing spatial comparison of virtual objects linked to a physical reference object. Accordingly, claim 1 integrates the alleged judicial exception into a practical application. For at least these reasons, Applicant asserts that claim 1 (and its dependents) is patent- eligible. Claims 9 and 17 include similar recitations as claim 1 and are patent-eligible at least for similar reasons. Accordingly, Applicant respectfully requests withdrawal of the rejections under 35 U.S.C. § 101.” to be persuasive. As amended, the claims now recite the limitations of “while presenting the representation of the selected virtual object at the comparison position, detecting a reorientation of the physical object; and in response to detecting the reorientation of the physical object, reorienting, via the display, the representation of the selected virtual object in accordance with the reorientation of the physical object.” which recite more than merely displaying information on a display, but rather update of the display of a virtual object in a comparison position defined relative to the position of the real-world object in response to the reorientation of the physical object and therefore integrate the judicial exception into a practical application by applying or using the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception - see MPEP 2106.05(e) and Vanda Memo. For at least these reasons, the claims are no longer rejected under 35 USC 101.
With respect to the remarks filed 3/23/2026 directed to the prior art rejection, the rejection under 35 USC 102 has been withdrawn and the claims as amended are now rejected under 35 USC 103 over Ayush in view of Faranda. The remarks have been considered in view of the claim amendments and updated rejection. While Ayush and Bronickius are still relied upon in the overall rejection of the claims, the combination is not relied upon for the amended portion of the claims as discussed in the interview on 2/24/2026 and in the remarks filed 3/23/2026. Claims 1-24 remain rejection under 35 USC 103.
Relevant Art Not Cited
US 2015/0309578 discloses displaying information about a real world item in a virtual environment.
In-store AR Enabled Product Comparison and Recommendation shows the display of a real world item with recommended corresponding products for the initial item.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VICTORIA E. FRUNZI whose telephone number is (571)270-1031. The examiner can normally be reached Monday- Friday 7-4 (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Marissa Thein can be reached at (571) 272-6764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
VICTORIA E. FRUNZI
Primary Examiner
Art Unit TC 3689
/VICTORIA E. FRUNZI/Primary Examiner, Art Unit 3689 5/5/2026
Prosecution Insights