DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 02/04/2025 and 11/17/2025 were filed after the mailing date of the application. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claim 2 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 of U.S. Patent No. 12,254,049 (Patent ‘049, hereinafter). Although the claims at issue are not identical, they are not patentably distinct from each other because all of the features of claim 2 of the current application are already included in claim 1 of Patent ‘049. See tables below.
Table I:
Current Application: 19/045311
U.S. Patent No. 12,254,049
2
1
Table II:
Current Application: 19/045311
U.S. Patent No. 12,254,049
2. A method comprising:
obtaining one or more tags associated with a subset of augmented reality (AR) experiences;
determining that a set of tags of the one or more tags associated with an individual AR experience of the subset of AR experiences corresponds to a blocked category; and
removing the individual AR experience from the subset of AR experiences in response to determining that the set of tags associated with the individual AR experience corresponds to the blocked category, further comprising:
receiving an image from a device;
applying a machine learning model to the image to generate an embedding query vector, the machine learning model trained to encode a plurality of images and text into a common embedding space;
searching, based on the embedding query vector, a database of AR experiences to identify the subset of AR experiences associated with one or more embeddings that correspond to the embedding query vector, the subset of AR experiences being identified by:
computing a Gaussian distance metric between the embedding query vector and each of a plurality of indices; and
determining that nearest neighbor indices are associated with a Gaussian distance metric that corresponds to a similarity threshold; and
transmitting to the device the subset of AR experiences associated with the one or more embeddings that correspond to the embedding query vector.
1. A method comprising:
receiving, by one or more processors, an image from a device;
applying a machine learning model to the image to generate an embedding query vector, the machine learning model trained to encode a plurality of images and text into a common embedding space;
searching, based on the embedding query vector, a database of augmented reality (AR) experiences to identify a subset of AR experiences associated with one or more embeddings that correspond to the embedding query vector, the subset of AR experiences being identified by:
computing a Gaussian distance metric between the embedding query vector and each of a plurality of indices; and
determining that nearest neighbor indices are associated with a Gaussian distance metric that corresponds to a similarity threshold; and
transmitting to the device the subset of AR experiences associated with the one or more embeddings that correspond to the embedding query vector.
From the tables above, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify claim 1 of Patent ‘049 to be as current claim 2, since all the features of claim 2 are already included in claim 1 of Patne 049.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 6-10, 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Bimbra et al. (US. Patent App. Pub. No. 2009/0115776, “Bimbra”, hereinafter) in view of Hunter et al. (US. Patent App. Pub. No. 2012/0221687, “Hunter”).
As per claim 1, Bimbra teaches a method comprising:
obtaining one or more tags associated with a subset of augmented reality (AR) experiences (further addressed below, see Fig. 1, 2A-2C, ¶ [4], ¶ [28], ¶ [30-32], i.e., obtaining tags associated with individuals in an immersive virtual world);
determining that a set of tags of the one or more tags associated with an individual AR experience of the subset of AR experiences corresponds to a blocked category (¶ [32], “Content presented to users may be based on the users' tags, thereby limiting content to that appropriate to a particular user's role”); and
removing the individual AR experience from the subset of AR experiences in response to determining that the set of tags associated with the individual AR experience corresponds to the blocked category (¶ [41]).
Bimbra does not expressly teach the tags associated with augmented reality (AR) experience, but rather with an immersive virtual world experience.
However, Hunter teaches a very similar method of adding tags to media file that can be removed when inappropriate (see ¶ [86]), wherein the media file is an augmented reality experience (¶ [42]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method as taught by Hunter into the method as taught by Bimbra as addressed above, the advantage of which is to provide an enhanced sensory, educational or entertainment experience to a user (¶ [42]).
As per claim 3, the combined teachings of Bimbra and Hunter also comprise causing a device to modify an image based on an AR experience selected from a user interface that presents the subset of AR experiences (see Hunter, ¶ [53] referring to Fig. 7). Thus, claim 3 would have been obvious over the combined references for the reason above.
As per claim 6, the combined Bimbra-Hunter substantially teaches:
presenting a user interface to a first user of a plurality of users; receiving a textual search query from the first user via the user interface, the textual search query comprising a request to identify one or more AR experiences (Hunter, ¶ [72], ¶ [118]);
presenting, in the user interface, search results in response to receiving the textual search query, the search results comprising a plurality of AR experiences (Hunter, ¶ [48], ¶ [119]. See also Fig. 28 of Hunter);
monitoring interactions of the first user with the user interface to collect metadata that identifies the interactions indicating whether the first user has interacted with a given AR experience of the plurality of AR experiences (Hunter, Fig. 3, ¶ [86]); and
storing each of the interactions in the metadata together with the textual search query as part of a search log in association with the given AR experience (Hunter ¶ [66], storing user information along geo-tag media data). Thus, claim 6 would have been obvious over the combined references for the reason above.
As per claim 7, the combined Bimbra-Hunter further teaches ranking the subset of AR experiences based on one or more criteria (Hunter, ¶ [50]). Thus, claim 7 would have been obvious over the combined references for the reason above.
As per claim 8, the combined Bimbra-Hunter does impliedly teach accessing a search log that includes a plurality of search queries for AR experiences received previously from a plurality of users (Hunter, ¶ [86], ¶ [90], ¶ [101], accessing information about the user's history and previous interactions with the system);
identifying a given AR experience that was selected in response to a first search query of the plurality of search queries (Hunter, ¶ [43], “In one sense, a user's movement through the real world informs a search query for content that the user is likely to consider relevant or of interest”); and storing a first set of text of the first search query in association with the given AR experience (Hunter, Fig. 28, ¶ [158], storing search query in the last days, months, or years). Thus, claim 8 would have been obvious over the combined references for the reason above.
As per claim 9, as addressed in claim 8, the combined Bimbra-Hunter does also impliedly teach determining that the given AR experience was selected in response to a second search query of the plurality of search queries; and storing a second set of text of the second search query in association with the given AR experience (because search queries are stored for later use).
As per claim 10, as addressed above, the combined Bimbra-Hunter substantially teaches repeating the identifying and storing for each of the plurality of search queries to generate a database that associates each of a plurality of AR experiences with a respective set of text of the plurality of search queries (see Hunter, ¶ [44] in combination with the steps of identifying and storing addressed in claims 8 and 9 above). Thus, claim 10 would have been obvious over the combined references for the reason above.
Claim 17, which is similar in scope to claim 1 as addressed above, is thus rejected under the same rationale.
As per claim 18, the combined Bimbra-Hunter does teach wherein a device generates a user interface that presents the subset of AR experiences (Hunter, Fig. 28). Thus, claim 18 would have been obvious over the combined references for the reason above.
Claim 19, which is similar in scope to claim 1 as addressed above, is thus rejected under the same rationale.
Claim 20, which is similar in scope to claim 18 as addressed above, is thus rejected under the same rationale.
Claims 4, 5, 15, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Bimbra et al. (US. Patent App. Pub. No. 2009/0115776) in view of Hunter et al. (US. Patent App. Pub. No. 2012/0221687), further in view of Sreenivasan et al. (US. Patent App. Pub. No. 2022/0020482, “Sreenivasan”).
As per claim 4, the combined Bimbra-Hunter does not expressly teach identifying nearest neighbor indices for an embedding query vector; and selecting the subset of AR experiences in response to determining that the subset of AR experiences are associated with the nearest neighbor indices.
However, in a similar method of generating AR experience (¶ [8-9]), Sreenivasan teaches this feature, i.e., identifying nearest neighbor indices for an embedding query vector; and selecting the subset of AR experiences in response to determining that the subset of AR experiences are associated with the nearest neighbor indices (see ¶ [43], “In one embodiment, content of the database 26 is indexed by index vectors generated using a same process as is used in generating the one or more query vectors. The matching includes matching the one or more query vectors with the index vectors indexing the retrieved AR instructional content”. It is noted that how nearest neighbor indices are determined (i.e., either spatially or logically); therefore, it is interpreted as the matching of indexes).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the method as taught by Sreenivasan to the combined Bimbra-Hunter as addressed above, the advantage of which is to provide real-time automated retrieval of augmented reality (AR) content to the user (¶ [12]).
As per claim 5, the combined Bimbra-Hunter-Sreenivasan implicitly teach computing a cosine similarity metric between the embedding query vector and each of a plurality of indices; and determining that the nearest neighbor indices are associated with the cosine similarity metric that corresponds to a similarity threshold (see Sreenivasan, ¶ [43], determining the matching between AR instructional content and the generated query vectors using cosine similarity metric). Thus, claim 5 would have been obvious over the combined references for the reason above.
As per claim 15, the combined Bimbra-Hunter-Sreenivasan further teaches training a machine learning model by performing operations comprising:
receiving training data comprising a plurality of image-text pairs; selecting a first image-text pair from the plurality of image-text pairs (Sreenivasan, ¶ [32-33]);
applying the machine learning model to the first image-text pair to relate visual embeddings of an image of the first image-text pair and textual embeddings of text of the first image-text pair with a same embedding in a common embedding space (Sreenivasan, ¶ [33], training image-text links. See also, ¶ [25]); and
repeating the selecting and applying operations for each of the plurality of image-text pairs (impliedly included and pre-stored in the service manual under image-text links taught by Sreenivasan at ¶ [33]). Thus, claim 15 would have been obvious over the combined references for the reason above.
As per claim 16, as addressed in claim 15, the combined Bimbra-Hunter-Sreenivasan does teach wherein a machine learning model comprises a network that indicates correspondence between a given image and given text (Sreenivasan, ¶ [25], ¶ [33]).
Claims 11 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Bimbra et al. (US. Patent App. Pub. No. 2009/0115776) in view of Hunter et al. (US. Patent App. Pub. No. 2012/0221687), further in view of Collomosse et al. (US. Patent App. Pub. No. 2022/0092108, “Collomosse”).
As per claim 11, the combined Bimbra-Hunter does not explicitly teach applying a machine learning model to a set of text associated with each of the plurality of AR experiences of the database to generate an AR experience embedding for each of the plurality of AR experiences.
However, Collomosse teaches a similar method of generating database for AR experiences from search queries (¶ [32]), wherein the method further includes the above features, i.e., applying a machine learning model to a set of text associated with each of the plurality of AR experiences of the database to generate an AR experience embedding for each of the plurality of AR experiences (¶ [39], generates a style embedding that includes a feature vector of one or more constituent style codes combined together. See further ¶ [29] and Fig. 4, ¶ [54-55] for further teachings of applying a machine learning model to the image to generate an embedding query vector).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method as taught by Collomosse into the combined Bimbra-Hunter as addressed above, the advantage of which is to provide accurate digital image search (¶ [28]).
As per claim 12, although not explicitly taught by the combined Bimbra-Hunter, Collomosse does teach wherein searching the database of AR experiences comprising searching the AR experience embedding for each of the plurality of AR experiences to identify the subset of AR experiences that are associated with AR experience embeddings ((¶ [75], “Using the combined style embedding 612, the style search system 102 identifies digital images with similar style 614”. Please refer to claim 10 for the generating of database of AR experiences). Thus, claim 12 would have been obvious over the combined references for the reason above.
Claims 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Bimbra et al. (US. Patent App. Pub. No. 2009/0115776) in view of Hunter et al. (US. Patent App. Pub. No. 2012/0221687), and further in view of Stoop et al. (US. Patent App. Pub. No. 2018/0089542, “Stoop”).
As per claim 13, the combined Bimbra-Hunter-Collomosse does not expressly teach removing non-ASCII letter search queries from the plurality of search queries.
However, Stoop, in a similar method of query search (see Abstract), teaches this feature, i.e., removing non-ASCII letter search queries from the plurality of search queries (¶ [57], “…such as removing certain numbers and punctuation (including the “#” character in a hashtagged term), removing or replacing special characters and accents, lower-casing all text, other suitable pre-processing steps, or any combination thereof”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the method as taught by Stoop to the combined Bimbra-Hunter-Collomosse method, the advantage of which is to eliminate insignificant search terms (¶ [57]).
As per claim 14, although not explicitly taught by the combined Bimbra-Hunter-Collomosse, Stoop does impliedly teach removing search queries from the plurality of search queries that fail to correspond to a length criteria (also ¶ [57] of Stoop, removing insignificant terms from the search query). Thus, claim 14 would have been obvious over the combined references for the reason above.
Allowable Subject Matter
Claim 2 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: The prior art taken singly or in combination does not teach or suggest, a method, among other things, comprising:
…the subset of AR experiences being identified by:
computing a Gaussian distance metric between the embedding query vector and each of a plurality of indices; and
determining that nearest neighbor indices are associated with a Gaussian distance metric that corresponds to a similarity threshold; and
transmitting to the device the subset of AR experiences associated with the one or more embeddings that correspond to the embedding query vector.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hau H. Nguyen whose telephone number is: 571-272-7787. The examiner can normally be reached on MON-FRI from 8:30-5:30.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard, can be reached on (571) 272-7773.
The fax number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/HAU H NGUYEN/Primary Examiner, Art Unit 2611
Prosecution Insights