DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant claims continuation priority to application 18,470,818, now Patent No. 12,266,010, filed 9/20/2023, which claims continuation priority to application 17/711,536, now Patent No. 11,842,385, filed 1/4/2022, which claims continuation priority to application 16/229,369, now Patent No. 11,321,768, filed 12/21/2018.
Information Disclosure Statement
The IDSs submitted on 2/24/2025 has been considered.
Status of Claims
Applicant’s claims, filed 2/24/2025, have been entered. Claims 1-21 are currently pending in this application and have been examined.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-21 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-24 of U.S. Patent No. 11,321,768 B2.
Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the patent anticipate the claims of the present application.
Claims 1-21 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 11,842,385 B2.
Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the patent anticipate the claims of the present application.
Claims 1-21 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 12,266,010 B2.
Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the patent anticipate the claims of the present application.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) an abstract idea. This judicial exception is not integrated into a practical application. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Under Step 1 of the Alice/Mayo test the claims are directed to statutory categories. Specifically, the method, as claimed in claims 1-10, are directed to a process, the system, as claimed in claims 11-20, are directed to a machine, and the non-transitory computer readable medium, as claimed in claim 21, is directed to an article of manufacture (see MPEP 2106.03).
Under Step 2A (prong 1), claim 1, taken as representative, recites at least the following limitations (emphasis added) that recite an abstract idea:
outputting, for display, content in which a object is overlaid on a user's real-world environment at a first location;
receiving an indication of an amount by which the object should be moved; and
outputting, for display, updated content in which the object is instead overlaid on the user's real-world environment at a second location, wherein the second location is determined using the indication received of the amount by which the object should be moved.
These limitations recite certain methods of organizing human activity, such as performing commercial interactions (see MPEP 2106.04(a)(2)(II)). Certain methods of organizing human activity are defined by MPEP 2106.04 as including “fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions).” In this case, the abstract ideas recited in representative claim 1 are certain methods of organizing human activity because outputting content for display and outputting updated content for display based on an amount by which the object should be moved is a managing personal behavior or relationships or interactions between people because it is following rules or instructions. Thus, claim 1 recites an abstract idea.
Independent claims 11 and 21 recite the same abstract idea as recited in independent claim 1. As such, the analysis under Step 2A, Prong 1 is the same for independent claims 11 and 21 as described above for independent claim 1.
Under Step 2A (prong 2), if it is determined that the claims recite a judicial exception, it is then necessary to evaluate whether the claims recite additional elements that integrate the judicial exception into a practical application of that exception (see MPEP 2106.04). As stated in the MPEP, when “an additional element merely recites the words ‘apply it (or an equivalent) with the judicial exception, or merely uses a computer as a tool to perform an abstract idea,” the judicial exception has not been integrated into a practical application.
In this case, claim 1 includes additional elements such as (additional elements are bolded):
A computer-implemented method comprising:
outputting, for display on a first computing device, augmented reality (AR) content in which a virtual object is overlaid on a user's real-world environment at a first location;
receiving, from a second computing device, an indication of an amount by which the virtual object should be moved; and
outputting, for display on the first computing device, updated AR content in which the virtual object is instead overlaid on the user's real-world environment at a second location, wherein the second location is determined using the indication received from the second computing device of the amount by which the virtual object should be moved.
In this case, claim 11 includes additional elements such as (additional elements are bolded):
A system comprising:
a processor;
a memory coupled to the processor, the memory storing instructions which, when executed by the processor, cause the system to:
output, for display on a first computing device, augmented reality (AR) content in which a virtual object is overlaid on a user's real-world environment at a first location;
receive, from a second computing device, an indication of an amount by which the virtual object should be moved; and
output, for display on the first computing device, updated AR content in which the virtual object is instead overlaid on the user's real-world environment at a second location, wherein the second location is determined using the indication received from the second computing device of the amount by which the virtual object should be moved.
In this case, claim 21 includes additional elements such as (additional elements are bolded):
A non-transitory computer-readable medium containing instructions which, when executed by a computer, cause the computer to:
output, for display on a first computing device, augmented reality (AR) content in which a virtual object is overlaid on a user's real-world environment at a first location;
receive, from a second computing device, an indication of an amount by which the virtual object should be moved; and
output, for display on the first computing device, updated AR content in which the virtual object is instead overlaid on the user's real-world environment at a second location, wherein the second location is determined using the indication received from the second computing device of the amount by which the virtual object should be moved.
Although reciting these additional elements, taken alone or in combination these elements are not sufficient to integrate the abstract idea into a practical application. These additional elements merely amount to the general application of the abstract idea to a technical environment (“computer” implemented, “on a first computing device”, “augmented reality (AR)” content, “from a second computing device”, “virtual” content, “a processor”, “a memory coupled to the processor, the memory storing instructions”, “a non-transitory computer-readable medium containing instructions”) and insignificant pre-and-post solution activity (outputting/transmitting information, displaying information, receiving information). The specification makes clear the general-purpose nature of the technological environment. This is because the additional elements of claims 1, 11, and 21 are recited at a high level of generality (i.e., as generic computing hardware) such that they amount to nothing more than the mere instructions to implement or apply the abstract idea on generic computing hardware (or, merely uses a computer as a tool to perform the abstract idea) (paragraphs [0027]-[0028], [0063], [0089]-[0103]). The specification indicates that while exemplary general-purpose systems may be specific for descriptive purposes, any elements capable of implementing the claimed invention are acceptable. That is, the technology used to implement the invention is not specific or integral to the claim. The description demonstrates that these additional elements are merely generic devices such as a generic computer. Further, the additional elements do no more than generally link the use of a judicial exception to a particular environment or field of use (such as the Internet or computing networks).
Therefore, considered both individually and as an ordered pair, the additional elements do no more than generally link the use of the abstract idea to a particular technological environment or field of use. That is, given the generality with which the additional elements are recited, the limitations do not implement the abstract idea with, or use the abstract idea in conjunction with, a particular machine or manufacture that is integral to the claim. Additionally, the claims do not reflect an improvement in the functioning of a computer, or an improvement to other technology or technical field, do not transform or reduction of a particular article to a different state or thing; and do not apply or use the abstract idea in some other meaningful way beyond generally linking the use of the abstract idea to a particular technology environment, such that the claim as a whole is more than a drafting effort designed to monopolize the abstract idea into a practical application, and is therefore “directed to” the abstract idea.
In addition to the above, the recited outputting/transmitting, displaying, receiving steps (even assuming arguendo they do not form part of the abstract idea, which the Examiner does not acquiesce), are at best little more than extra-solution activity (e.g., data gathering, presentation of data) that contributes nominally or insignificantly to the execution of the claimed system (see MPEP 2106.05(g)).
In view of the above, under Step 2A (prong 2), claims 1, 11, and 21 do not integrate the recited exception into a practical application.
Under Step 2B, examiners should evaluate additional elements individually and in combination to determine whether they provide an inventive concept (i.e., whether the additional elements amount to significantly more than the exception itself). In this case, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Returning to claims 1, 11, and 21, taken individually or as a whole the additional elements of claims 1, 11, and 21 do not provide an inventive concept (i.e. they do not amount to “significantly more” than the exception itself). As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements used to perform the claimed process amount to no more than the mere instructions to apply the exception using a generic computer and/or no more than a general link to a technological environment.
Furthermore, the additional elements fail to provide significantly more also because the claim simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception. For example, the additional elements of claims 1, 11, and 21 utilize operations the courts have held to be well-understood, routine, and conventional (see: MPEP 2106.05(d)(II)), including at least:
receiving or transmitting data over a network,
storing or retrieving information from memory,
presenting offers
Even considered as an ordered combination (as a whole), the additional elements of claims 1, 11, and 21 do not add anything further than when they are considered individually.
In view of the above, representative claims 1, 11, and 21 do not provide an inventive concept (“significantly more”) under Step 2B, and is therefore ineligible for patenting.
Regarding claims 2, 5-10, 12, and 15-20
Dependent claim(s) 2, 5-10, 12, and 15-20, when analyzed as a whole, are held to be patent ineligible under 35 U.S.C. 101 because they do not add “significantly more” to the abstract idea. More specifically, dependent claim(s) 2, 5-10, 12, and 15-20 merely further define the abstract limitations of claim(s) 1 and 11 or provide further embellishments of the limitations recited in independent claim claim(s) 1 and 11.
Claims 2, 5-10, 12, and 15-20 set forth:
wherein the indication of the amount by which the virtual object should be moved comprises a coordinate indicative of the second location.
wherein the indication of the amount by which the virtual object should be moved comprises an indication of a position with respect to an object or reference point.
wherein the indication of the amount by which the virtual object should be moved comprises a delta value corresponding to an amount the virtual object should be moved in one or more dimensions with respect to the first location in a coordinate system.
wherein an indicated change in location of the virtual object is received from the second computing device in the form of the indication, without receiving from the second computing device an AR image in which the virtual object is overlaid on the user's real-world environment at the second location.
wherein prior to outputting the AR content in which the virtual object is overlaid on the user's real-world environment at the first location, the method comprises: obtaining one or more images captured by the first computing device that depict the user's real-world environment; determining 3D feature data corresponding to the one or more images; obtaining an indication of the virtual object; determining a size, orientation, and position of the virtual object based on the first location and based on the 3D feature data; and generating the AR content by overlaying the virtual object, having the determined size, orientation, and position, at the first location.
wherein prior to outputting the AR content in which the virtual object is overlaid on the user's real-world environment at the first location, the method further comprises: receiving, from the second computing device, a positioning signal indicative of a positioning of the virtual object that corresponds to the first location.
further comprising receiving new size information of the virtual object, from the second computing device along with the indication, and wherein the virtual object is overlaid on the user's real-world environment at the second location at the new size.
Such recitations merely embellish the abstract idea of outputting content for display and outputting updated content for display based on an amount by which the object should be moved. The claims do not set forth any further additional limitations, and therefore such abstract embellishments are applied to the additional limitations recited in claim(s) 1 and 11, which do no more than generally link the use of the abstract idea to a particular technological environment, do not integrate the abstract idea into a practical application, and do not provide an inventive concept. Accordingly, the claims do not confer eligibility on the claimed invention and is ineligible for similar reasons to claim(s) 1 and 11.
Thus, dependent claims 2, 5-10, 12, and 15-20 are ineligible.
Regarding claim 3, 4, 13, and 14
Dependent claim(s) 3, 4, 13, and 14 sets forth:
wherein the coordinate is a 2-D screen coordinate associated with a location corresponding to a selection on a screen of the second computing device.
further comprising receiving an identification of the virtual object and an instruction to move the virtual object, from the second computing device along with the indication.
Such recitations merely embellish the abstract idea of outputting content for display and outputting updated content for display based on an amount by which the object should be moved. While the claim(s) do set forth the additional elements of “a screen of the second computing device”, these recitations are similar to the additional limitations in claims 1 and 11, as they do no more than generally link the use of the abstract idea to a particular technological environment. That is these additional elements merely amount to the general application of the abstract idea to a technical environment. The specification makes clear the general-purpose nature of the technological environment and indicates that while exemplary general-purpose systems may be specific for descriptive purposes, any elements capable of implementing the claimed invention are acceptable. That is, the technology used to implement the invention is not specific or integral to the claim. Therefore, these additional elements do not integrate the abstract idea into a practical application because they merely amount to using a computer to apply the abstract idea and no more than a general link of the use of the abstract idea to a particular technological environment or field of use and thus do not act to integrate the abstract idea into a practical application of the abstract idea.
Additionally, the additional elements do not amount to significantly more because they merely amount to using a computer to apply the abstract idea and amount to no more than a general link of the use of the abstract idea to a particular technological environment.
Thus, dependent claims 3, 4, 13, and 14 are also ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zia et al. (US 2019/0197599 A1) in view of Hoover et al. (US 2018/0045963 A1).
Regarding claim 1, Zia et al., hereinafter Zia, discloses a computer-implemented method comprising:
outputting, for display on a first computing device, augmented reality (AR) content in which a virtual object is overlaid on a user's real-world environment at a first location (Figs. 4 and 7; ¶0045 [the information about the products and their position in virtual space (AR space) are communicated to the AR application executing on the end-user's mobile computing device, where, when the end-user views the space using the AR application, the live AR view of the space will include the presentation of the products, as selected and positioned, by the room designer using the room design application] and ¶0055 [FIG. 7 is a user interface diagram illustrating an example of a user interface for an AR-capable mobile application that provides for viewing AR scenes, for which a coffee table has been selected and positioned by one or more remote room designers] in view of ¶0015 [the term “end-user” will be used in reference to a potential consumer or person who is operating an AR-capable mobile computing device, executing an AR mobile application, in some physical space (e.g., a room) in an effort to visualize that space, enhanced with some visual content. Similarly, for purposes of the present disclosure, the term “room designer” is used in reference to a person operating a room design application, typically executing on a client computing device, that is remote from the end-user.], ¶0016 [the AR metadata allows the AR mobile application to “pin” objects to a position within the virtual space, so that images of those objects can be properly rendered (e.g., position, skew and size) as superimposed over images of the physical space, thereby giving the end-user the perception that the objects (e.g., superimposed images) are physically present in the physical space], ¶0018 [by interacting with the room design application, the room designer can select and position products within images, as presented on his or her client computing device, such that the positioned products will also appear in a live AR scene as rendered by the end-user's AR-capable mobile computing device], and ¶0035 [when a room designer interacts with the images by positioning an image of a product, the image is “tied” to the virtual space (AR space), as represented by the AR metadata, and thus, when the corresponding object is rendered in AR space on the end-user's mobile computing device, the object will maintain its position and orientation relative to the real-world scene]);
receiving, from a second computing device, an indication by which the virtual object should be moved (Figs. 6-7; ¶0011 [FIG. 6 is a user interface diagram illustrating an example of a user interface for a design application, via which a room designer selects and positions objects, such that a visual representation of those objects will appear in a live AR view at an end-user's AR-capable mobile computing device], ¶0044 [Using the design application, the room designer will manipulate the images of the recommended products by positioning the images of the products, overlaid on one or more of the images of the physical space. By positioning the images in this manner, the room designer is identifying the position of the products in virtual space, and thus effecting how—specifically, where—the images of the product will appear when rendered on the end-user's AR-capable mobile computing device. Accordingly, at method operation 414, the server receives an indication of a selection and position placement of some set of products, of which, some may be from the set of candidate products. The position information derived from the placement of the images via the user interface presented to the room designer is stored in association with images of the products, at the server.], ¶¶0055-0056 [the end-user can discuss, in real time, his or her preferences with the room designer. If, for example, the room designer positions a coffee table in AR space, and the end-user does not like it, the end-user can simply indicate this preference to the room designer (e.g., via the live video call session)… the room designer can save different room designs, with different selections of products and/or different placements of those products in AR space, for a given end-user's design consultation request], ¶¶0034-0035 [the session management module 310 includes a live voice and video call session module 314. This module 314 processes requests for live connections between end-users and room designers, ensuring that the respective devices are connected for the purpose of having live voice or video calls] in view of ¶0027 [The room designer may generate one or more room designs, with each room design including a selection of one or more products, and information as to where the one or more products are to appear in AR space, in accordance with the image or images and AR metadata received from the end-user's mobile computing device]); and
outputting, for display on the first computing device, updated AR content in which the virtual object is instead overlaid on the user's real-world environment at a second location, wherein the second location is determined using the indication received from the second computing device by which the virtual object should be moved (Figs. 6-7; ¶0034 [any superimposed images may be updated to reflect different images that are consistent with what has been requested by the end-user] and ¶0050 [once a product has been positioned in an image, the information about the product and its positioning in AR space associated with the real-world physical space, is communicated to the end-user's mobile computing device…render the real-world scene including the superimposed image of the coffee table, positioned properly in AR space], ¶¶0055-0056 [the end-user can discuss, in real time, his or her preferences with the room designer. If, for example, the room designer positions a coffee table in AR space, and the end-user does not like it, the end-user can simply indicate this preference to the room designer (e.g., via the live video call session)… the room designer can save different room designs, with different selections of products and/or different placements of those products in AR space, for a given end-user's design consultation request] in view of ¶0016 [the AR metadata allows the AR mobile application to “pin” objects to a position within the virtual space, so that images of those objects can be properly rendered (e.g., position, skew and size) as superimposed over images of the physical space, thereby giving the end-user the perception that the objects (e.g., superimposed images) are physically present in the physical space], and ¶¶0018-0019 [collaborative live real-time AR viewing session], and ¶0035 [when a room designer interacts with the images by positioning an image of a product, the image is “tied” to the virtual space (AR space), as represented by the AR metadata, and thus, when the corresponding object is rendered in AR space on the end-user's mobile computing device, the object will maintain its position and orientation relative to the real-world scene]).
While Zia discloses a remote user indicating where a virtual object should be moved and updating the virtual space with the different placements of those products in the AR space (Figs. 6-7; ¶¶0055-0056), Zia does not explicitly disclose receiving an indication of an amount by which the virtual object should be moved and outputting updated AR content in which the virtual object is instead overlaid, wherein the second location is determined using the indication of the amount by which the virtual object should be moved. However, in the field of automatically repositing a virtual object with respect to a destination object in a three-dimensional (3D) environment of a user (abstract), Hoover et al., hereinafter Hoover teaches receiving an indication of an amount by which the virtual object should be moved and outputting updated AR content in which the virtual object is instead overlaid, wherein the second location is determined using the indication of the amount by which the virtual object should be moved (Figs. 15A-16; ¶¶0157-0164). The step of Hoover is applicable to the method of Zia as they share characteristics and capabilities, namely, they are directed to displaying objects in AR environments. It would have been obvious to one of ordinary skill in the art at the time of filing to modify the indication by which the virtual object should be moved as taught by Zia with the amount by which the virtual object should be moved as taught by Hoover. One of ordinary skill in the art at the time of filing would have been motivated to expand the method of Zia in order to express the orientation and location of the virtual object in vector form and update the values in the vectors as the virtual object moves around (Hoover ¶0159).
Regarding claim 2, Zia in view of Hoover teaches the computer-implemented method of claim 1. Hoover further teaches wherein the indication of the amount by which the virtual object should be moved comprises a coordinate indicative of the second location (Figs. 15A-16; ¶¶0157-0164). The motivation for making this modification to the teachings of Zia are the same as that set forth above, in the rejection of claim 1.
Regarding claim 3, Zia in view of Hoover teaches the computer-implemented method of claim 2. While Zia further discloses receiving an indication of a location corresponding to a selection on a screen of the second computing device (Figs. 6-7; ¶0059 [the display device (810), input device (812) and UI navigation device (814) may be a touch screen display], ¶0044 [Using the design application, the room designer will manipulate the images of the recommended products by positioning the images of the products, overlaid on one or more of the images of the physical space. By positioning the images in this manner, the room designer is identifying the position of the products in virtual space, and thus effecting how—specifically, where—the images of the product will appear when rendered on the end-user's AR-capable mobile computing device. Accordingly, at method operation 414, the server receives an indication of a selection and position placement of some set of products, of which, some may be from the set of candidate products. The position information derived from the placement of the images via the user interface presented to the room designer is stored in association with images of the products, at the server.]), Zia does not explicitly disclose wherein the coordinate is a 2-D screen coordinate associated with a location corresponding to a selection on a screen of the second computing device. However, Hoover further teaches wherein the coordinate is a 2-D screen coordinate associated with a location corresponding to a selection on a screen of the second computing device (Hoover Figs. 15A-16; ¶0028 [2D or 3D], ¶0062 [the user input device 466 can include a… touchscreen], ¶¶0157-0164 [destination object expressed in a vector form]). It would have been obvious to one of ordinary skill in the art at the time of filing to modify the indication of a location as taught by Zia with the 2D screen coordinate as taught by Hoover. One of ordinary skill in the art at the time of filing would have been motivated to expand the method of Zia in order to express the orientation and location to move the target object in vector form (Hoover ¶0160).
Regarding claim 4, Zia in view of Hoover teaches the computer-implemented method of claim 3, Zia further discloses further comprising receiving an identification of the virtual object and an instruction to move the virtual object, from the second computing device along with the indication (Figs. 6-7; ¶¶0055-0056).
Regarding claim 5, Zia in view of Hoover teaches the computer-implemented method of claim 1. Hoover further teaches wherein the indication of the amount by which the virtual object should be moved comprises an indication of a position with respect to an object or reference point (Figs. 15A-16; ¶¶0157-0164). The motivation for making this modification to the teachings of Zia are the same as that set forth above, in the rejection of claim 1.
Regarding claim 6, Zia in view of Hoover teaches the computer-implemented method of claim 1. Hoover further teaches wherein the indication of the amount by which the virtual object should be moved comprises a delta value corresponding to an amount the virtual object should be moved in one or more dimensions with respect to the first location in a coordinate system (Figs. 15A-16; ¶¶0157-0164). The motivation for making this modification to the teachings of Zia are the same as that set forth above, in the rejection of claim 1.
Regarding claim 7, Zia in view of Hoover teaches the computer-implemented method of claim 1. While Zia further teaches wherein an indicated change in location of the virtual object is received from the second computing device in the form of the indication (Figs. 6-7; ¶¶0055-0056), Zia does not explicitly disclose without receiving from the second computing device an AR image in which the virtual object is overlaid on the user's real-world environment at the second location. However, Hoover further teaches calculating a distance between the target virtual object and the destination object based on their locations and automatically orient and attach the virtual object to the destination object without receiving from the second computing device an AR image in which the virtual object is overlaid on the user's real-world environment at the second location (Figs. 15A-16; ¶¶0157-0164). It would have been obvious to one of ordinary skill in the art at the time of filing to modify the indicated change of a location as taught by Zia with the distance between the virtual object and the destination object as taught by Hoover. One of ordinary skill in the art at the time of filing would have been motivated to expand the method of Zia in order to automatically orient and attach the virtual object to the destination object (¶0161).
Regarding claim 8, Zia in view of Hoover teaches the computer-implemented method of claim 1, Zia further discloses wherein prior to outputting the AR content in which the virtual object is overlaid on the user's real-world environment at the first location, the method comprises:
obtaining one or more images captured by the first computing device that depict the user's real-world environment (Fig. 1, Fig. 4, element 404, Fig. 5, and Fig. 7; ¶0019 [the image (or, images) and corresponding AR metadata, as captured by the end-user's AR-capable mobile computing device, may be communicated to and stored at a remote server. A room designer, using the room design application, may subsequently access the previously captured image(s) and AR metadata, select and position one or more products to appear in one or more images, and then save the resulting room design at the server computer], ¶0040 [the server receives and stores an image (or, images), and corresponding AR metadata, as captured or derived with a camera and other sensors on the end-user's mobile computing device. The AR metadata generally consists of a data representation of a virtual space that corresponds with a real-world physical space presented in the one or more images] in view of ¶0016 [Using the AR mobile application, an end-user will invoke an AR viewing session. During the AR viewing session, the AR-capable mobile computing device uses input received from a combination of one or more image sensors and motion sensors to generate a data representation of a virtual space that corresponds with the physical space in which the mobile computing device is being operated], ¶0017 [one or more images of a physical space (e.g., a room) in combination with the corresponding AR metadata for that physical space, as captured by the mobile computing device, are communicated to a remote server] in further view of Figs. 1 (Examiner notes both elements 102 and 106 display the room 100 augmented with a coffee table 104) and 6 (Examiner notes the user interface design application creates and displays augmented images of the customer images with selected items on the designer computing device); ¶0034 [any superimposed images may be updated to reflect different images that are consistent with what has been requested by the end-user], ¶0056 [the room designer can save different room designs with… different placements of those products in AR space (Examiner notes the designer displays the room design on their device (i.e., first computing device) saving an room design with different placements of the products in the AR space is comparable to receiving one or more customer images depicting a customer environment] in view of ¶0027 [The room designer may generate one or more room designs, with each room design including a selection of one or more products, and information as to where the one or more products are to appear in AR space, in accordance with the image or images and AR metadata received from the end-user's mobile computing device]);
determining 3D feature data corresponding to the one or more images (¶0003 [The camera and/or other sensors may capture additional information from the environment that allows the mobile computing device to determine its position, orientation and motion, relative to visually distinguishing features, objects and/or inferred planes (e.g., walls, floors or other flat surfaces) in the images being captured by the camera, and thereby manipulate the rendering (e.g., position, skew and size) of any superimposed images to maintain the proper perspective and scale. This makes the presentation of the superimposed images seem realistic, creating an illusion for the viewer that the objects represented by the superimposed images are physically present in the scene]; Examiner notes the additional captured information from the environment is comparable to 3D feature data as described in the Specification at paragraph [0061]);
obtaining an indication of the virtual object (Figs. 1, 6; ¶0015 [a room design application having a user interface via which a room designer selects and positions products, which ultimately causes images of those products to appear in the AR view of the physical space], ¶0017 [the room designer can select visual content associated with various products available from an online marketplace of products (e.g., furniture and related home furnishings)], ¶0021 [with reference to FIG. 1, the room designer has selected a particular coffee table from a list of recommended products]);
determining a size, orientation, and position of the virtual object based on the first location and based on the 3D feature data (Fig. 4, element 414, Fig. 3, Fig. 6; ¶0044 [Using the design application, the room designer will manipulate the images of the recommended products by positioning the images of the products, overlaid on one or more of the images of the physical space. By positioning the images in this manner, the room designer is identifying the position of the products in virtual space, and thus effecting how—specifically, where—the images of the product will appear when rendered on the end-user's AR-capable mobile computing device… at method operation 414, the server receives an indication of a selection and position placement of some set of products; Examiner notes the room designer identifying the position of the products in the virtual space is comparable to determining a first location], ¶0017 [the room designer can select visual content associated with various products available from an online marketplace of products (e.g., furniture and related home furnishings) and then position those products in relation to the virtual space represented by the AR metadata], ¶0021 [After selecting the coffee table, by interacting with the user interface presented via client computing device 106, the room designer has positioned the coffee table in the AR scene, and submitted the room design to the server], ¶0050 [The room designer has positioned the top recommended coffee table in the image shown in panel 602], ¶0050 [once a product has been positioned in an image, the information about the product and its positioning in AR space associated with the real-world physical space, is communicated to the end-user's mobile computing device…render the real-world scene including the superimposed image of the coffee table, positioned properly in AR space] in view of ¶0016 [the AR metadata allows the AR mobile application to “pin” objects to a position within the virtual space, so that images of those objects can be properly rendered (e.g., position, skew and size) as superimposed over images of the physical space, thereby giving the end-user the perception that the objects (e.g., superimposed images) are physically present in the physical space (Examiner notes “positioned properly” and “properly rendered (e.g., position, skew and size)” is comparable to determining an appropriate size and orientation) and ¶0035 [when a room designer interacts with the images by positioning an image of a product, the image is “tied” to the virtual space (AR space), as represented by the AR metadata, and thus, when the corresponding object is rendered in AR space on the end-user's mobile computing device, the object will maintain its position and orientation relative to the real-world scene]); and
generating the AR content by overlaying the virtual object, having the determined size, orientation, and position, at the first location (¶0050 [once a product has been positioned in an image, the information about the product and its positioning in AR space associated with the real-world physical space, is communicated to the end-user's mobile computing device…render the real-world scene including the superimposed image of the coffee table, positioned properly in AR space] in view of ¶0015 [the term “end-user” will be used in reference to a potential consumer or person who is operating an AR-capable mobile computing device, executing an AR mobile application, in some physical space (e.g., a room) in an effort to visualize that space, enhanced with some visual content. Similarly, for purposes of the present disclosure, the term “room designer” is used in reference to a person operating a room design application, typically executing on a client computing device, that is remote from the end-user.], ¶0016 [the AR metadata allows the AR mobile application to “pin” objects to a position within the virtual space, so that images of those objects can be properly rendered (e.g., position, skew and size) as superimposed over images of the physical space, thereby giving the end-user the perception that the objects (e.g., superimposed images) are physically present in the physical space], ¶0019 [the image (or, images) and corresponding AR metadata, as captured by the end-user's AR-capable mobile computing device, may be communicated to and stored at a remote server. A room designer, using the room design application, may subsequently access the previously captured image(s) and AR metadata, select and position one or more products to appear in one or more images, and then save the resulting room design at the server computer. The end-user can then recall the room design as saved on the remote server. Upon initiating a subsequent AR viewing session in the same physical space for which a room designer has saved a room design, the AR mobile application will need to perform a realignment procedure to ensure that the old AR metadata (e.g., as originally shared with the room designer) is consistent with the new AR metadata (e.g., as generated during the current live AR viewing session). Accordingly, this allows the end-user to view the positioned products in a live AR view, at some time subsequent to when the room designer created the room design.], and ¶0035 [when a room designer interacts with the images by positioning an image of a product, the image is “tied” to the virtual space (AR space), as represented by the AR metadata, and thus, when the corresponding object is rendered in AR space on the end-user's mobile computing device, the object will maintain its position and orientation relative to the real-world scene]).
Regarding claim 9, Zia in view of Hoover teaches the computer-implemented method of claim 1, Zia further discloses wherein prior to outputting the AR content in which the virtual object is overlaid on the user's real-world environment at the first location, the method further comprises: receiving, from the second computing device, a positioning signal indicative of a positioning of the virtual object that corresponds to the first location (Fig. 4, element 414, Fig. 3, Fig. 6; ¶0044 [Using the design application, the room designer will manipulate the images of the recommended products by positioning the images of the products, overlaid on one or more of the images of the physical space. By positioning the images in this manner, the room designer is identifying the position of the products in virtual space, and thus effecting how—specifically, where—the images of the product will appear when rendered on the end-user's AR-capable mobile computing device… at method operation 414, the server receives an indication of a selection and position placement of some set of products; Examiner notes the room designer identifying the position of the products in the virtual space is comparable to determining a first location], ¶0017 [the room designer can select visual content associated with various products available from an online marketplace of products (e.g., furniture and related home furnishings) and then position those products in relation to the virtual space represented by the AR metadata], ¶0021 [After selecting the coffee table, by interacting with the user interface presented via client computing device 106, the room designer has positioned the coffee table in the AR scene, and submitted the room design to the server], ¶0050 [The room designer has positioned the top recommended coffee table in the image shown in panel 602]).
Regarding claim 10, Zia in view of Hoover teaches the computer-implemented method of claim 1. While Zia further discloses receiving new information of the virtual object, from the second computing device along with the indication, and wherein the virtual object is overlaid on the user's real-world environment at the second location (Figs. 6-7; ¶0011 [FIG. 6 is a user interface diagram illustrating an example of a user interface for a design application, via which a room designer selects and positions objects, such that a visual representation of those objects will appear in a live AR view at an end-user's AR-capable mobile computing device], ¶0044 [Using the design application, the room designer will manipulate the images of the recommended products by positioning the images of the products, overlaid on one or more of the images of the physical space. By positioning the images in this manner, the room designer is identifying the position of the products in virtual space, and thus effecting how—specifically, where—the images of the product will appear when rendered on the end-user's AR-capable mobile computing device. Accordingly, at method operation 414, the server receives an indication of a selection and position placement of some set of products, of which, some may be from the set of candidate products. The position information derived from the placement of the images via the user interface presented to the room designer is stored in association with images of the products, at the server.], ¶¶0055-0056 [the end-user can discuss, in real time, his or her preferences with the room designer. If, for example, the room designer positions a coffee table in AR space, and the end-user does not like it, the end-user can simply indicate this preference to the room designer (e.g., via the live video call session)… the room designer can save different room designs, with different selections of products and/or different placements of those products in AR space, for a given end-user's design consultation request], ¶¶0034-0035 [the session management module 310 includes a live voice and video call session module 314. This module 314 processes requests for live connections between end-users and room designers, ensuring that the respective devices are connected for the purpose of having live voice or video calls] in view of ¶0027 [The room designer may generate one or more room designs, with each room design including a selection of one or more products, and information as to where the one or more products are to appear in AR space, in accordance with the image or images and AR metadata received from the end-user's mobile computing device]), Zia does not explicitly teach receiving new size information of the virtual object, from the second computing device along with the indication, and wherein the virtual object is overlaid on the user's real-world environment at the second location at the new size. However, Hoover further teaches physical attributes assigned to the target virtual object comprise a size (¶0247) which can be modified based upon a focus indicator indicating a position of the target object (¶0110). It would have been obvious to one of ordinary skill in the art at the time of filing to modify the received indication as taught by Zia with the new size information as taught by Hoover. One of ordinary skill in the art at the time of filing would have been motivated to expand the method of Zia in order to modify the displayed object based on a perceived size or depth change (e.g., causing the target object to appear closer and/or larger when selected) (¶0110).
Regarding claims 11 and 21, the claims disclose substantially the same limitations, as claim 1, except claim 1 is directed to a process while claim 11 is directed to a system and claim 21 is directed to an article of manufacture. The added elements of “a processor and a memory coupled to the processor, the memory storing instructions” and “a non-transitory computer-readable medium containing instructions which, when executed by a computer” is also taught by Zia (Fig. 8; ¶¶0057-0063). Therefore, claims 11 and 21 are rejected for the same rational over the prior art cited in claim 1.
Regarding claims 12-20, the claims disclose substantially the same limitations, as claims 2-10, respectively, except claims 2-10 are directed to processes depending from claim 1 while claims 12-20 are directed to systems depending from claim 11. All limitations as recited have been analyzed and rejected with respect to claims 2-10, respectively, and do not introduce any additional narrowing of the scopes of the claims as analyzed. Therefore, claims 12-20 are rejected for the same rational over the prior art cited in claims 2-10.
Examiner’s Comment
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Reference U of the Notice of References Cited “Bringing Online Shopping Experience to Offline
Retail through Augmented Reality and RFID” discloses using AR to offline retail allowing users to browse products in a simulated shop that simulates the environment of a real shop.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LINDSEY B SMITH whose telephone number is (571)272-0519. The examiner can normally be reached Monday - Friday 9-6 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jeff Smith can be reached at 571-272-6763. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
LINDSEY B. SMITH
Examiner
Art Unit 3688
/LINDSEY B SMITH/Examiner, Art Unit 3688
/Jeffrey A. Smith/Supervisory Patent Examiner, Art Unit 3688
Prosecution Insights