DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 12/272,007. Although the claims at issue are not identical, they are not patentably distinct from each other because each provides addition of AR elements for association with a position relative to a real world object in an AR experience.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention at the time the invention was made to conclude that the invention defined in the claims at issue would have been an obvious variation of the invention defined in a claim in the patent because the patent enables storing and access of the AR elements for use in a launched second AR experience by a second device that displays the stored AR elements in a same position with a second image.
The following table shows the claims of the current application being examined and the conflicting claims of Patent 12/272,007.
Application: 19/056,296 Patent: 12/272,007
1
1
2
1, 2
3-20
3-20
Application: 19/056,296 Patent: 12/272,007
A method comprising: transmitting, to a remote server, from a first device, a request to access a first augmented reality (AR) experience that is part of a list of related AR experiences, the first AR experience being developed, by a second device in communication with the remote server, using a graphical user interface comprising a first option associated with a first AR element of a list of AR elements, the first option causing display positioning information of the first AR element to be stored after termination of the first AR experience on the first device, the graphical user interface comprising a second option associated with a second AR element of the list of AR elements, the second option preventing storage of display positioning information of the second AR element after termination of the first AR experience on the first device;
A method comprising: transmitting, to a remote server, from a first device, a request to access a first augmented reality (AR) experience that is part of a list of related AR experiences, the first AR experience having a first set of AR objects different from a second set of AR objects of a second AR experience in the list of related AR experiences, the first AR experience being developed, by a second device in communication with the remote server, using a graphical user interface comprising a first option associated with a first AR element of a list of AR elements, the first option causing display positioning information of the first AR element to be stored after termination of the first AR experience on the first device, the graphical user interface comprising a second option associated with a second AR element of the list of AR elements, the second option preventing storage of display positioning information of the second AR element after termination of the first AR experience on the first device
adding, by the first device, one or more AR elements of the first AR experience to a first image captured by the first device, the first image depicting a real-world object, the first device activating a camera to capture the first image that is used by the first AR experience to add the one or more AR elements
adding, by the first device, one or more AR elements of the first set of AR objects of the first AR experience to a first image captured by the device, the first image depicting a real-world object, the first device activating a camera to capture the first image that is used by the first AR experience to add the one or more AR elements
And in response to detecting selection of the first option by the second device during development of the first AR experience, causing storing, by the first device, on which the first AR experience is accessed, data representing a position of the one or more AR elements of the first AR experience relative to the real-world object, the data being maintained after the first AR experience is terminated
in response to detecting selection of the first option by the second device during development of the first AR experience, causing storing, by the first device, on which the first AR experience is accessed, data representing a position of the one or more AR elements of the first AR experience relative to the real-world object, the data being maintained after the AR experience is terminated;
receiving a request to launch the second AR experience after the first AR experience has been terminated
and in response to receiving the request to launch the second AR experience, accessing the data representing the position of the one or more AR elements of the first AR experience that was stored prior to termination of the first AR experience to generate a display of the second set of AR objects of the second AR experience, the second set of AR objects being placed at a particular position within a second image corresponding to the position of the one or more AR elements of the first set of AR objects of the first AR experience.
*Bold type indicates differences in the claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-8, 10, 13, and 16-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kyle Yoon et al., US 2021/0407151 A1 in view of Robert Auten et al., US 2017/0365098 A1 and further in view of Ryan McTaggart et al., US 2017/0084082 A1.
Independent claim 1, Yoon discloses a method comprising:
transmitting, to a remote server, from a first device, a request to access a first augmented reality (AR) experience, (i.e. accessing differing AR effects for presentation – abstract; computing system 100 can include multiple computing devices such as a headset and a core processing component, such as a server system that processes operations – Para 24);
adding, by the first device, one or more AR elements of the first AR experience to a first image captured by the device, the first image depicting a real-world object, the first device activating a camera to capture the first image that is used by the first AR experience to add the one or more AR elements (i.e. adding AR effects to a video depicting a person – Para 95; Fig. 10A – captured via camera – Para 22);
Yoon fails to disclose transmitting, to a remote server, from a first device, a request to access a first augmented reality (AR) experience that is part of a list of related AR experiences, the first AR experience having a first set of AR objects different from a second set of AR objects of a second AR experience in the list of related AR experiences, which Auten discloses (i.e. experience components, executed by a processor, receive requests from client devices to access an AR experience– Para 31, 33-34, 77; groups of AR experiences are stored and accessed by the experience components – Para 27, 77).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention at the time invention was made to combine Auten’s known method of receiving, from a device, a request to access a first augmented reality (AR) experience that is part of a list of related AR experiences, the first AR experience having a first set of AR objects different from a second set of AR objects of a second AR experience in the list of related AR experiences with the method of Yoon because selective access to AR objects that maintain persistent location provides the advantage of maintaining a common view of content displayed at differing times and improves sharing AR experiences among multiple device users (Auten, Para 66).
Yoon in view of Auten additionally fails to disclose the first AR experience being developed using a graphical user interface comprising a first option associated with a first AR element of a list of AR elements, the first option causing location data of the first AR element to be stored after termination of the first AR experience, the graphical user interface comprising a second option associated with a second AR element of the list of AR elements, the second option preventing storage of location data of the second AR element after termination of the first AR experience, which McTaggart discloses (i.e. a server storing AR experiences to be provided to a client device – Fig. 1A; the server unlocks an AR experience file enabling the stored filed to be edited or overwritten – Para 82; the server locks an AR experience associated with a target file to keep the stored data consistent – Para 79, 81; the server causes a client device to display an interface – Para 46 – that enables the storing, edit, deleting AR experiences – Para 40).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention at the time invention was made to combine McTaggart’s method of a first AR experience being developed using a graphical user interface comprising a first option associated with a first AR element of a list of AR elements, the first option causing location data of the first AR element to be stored after termination of the first AR experience, the graphical user interface comprising a second option associated with a second AR element of the list of AR elements, the second option preventing storage of location data of the second AR element after termination of the first AR experience with the method of Yoon in view of Austen because providing options to store AR elements enables validation of an AR experience with system requirements, which provides the benefit of improving quality of the AR system (McTaggart, Para 85).
Claim 2, Yoon discloses the method of claim 1, wherein the data is stored on a remote server associated with the device (i.e. computing device communicates with a server – Para 24, 28; computing device includes storage memory implemented in server – Para 42; Fig. 4), further comprising:
storing data representing a position of the one or more AR elements of the first AR experience relative to the real-world object, the data being maintained after the AR experience is terminated (i.e. storing the implementation parameters including the position of the AR effects, after ending display of the AR effects as applied to a live video – Para 59, 60);
receiving a request to launch the second AR experience after the first AR experience has been terminated (i.e. after ending display of a first selected AR effect a second process is initiated to allow display of AR effects to pre-recorded video – Para 67; Fig. 7); and
in response to receiving the request to launch the second AR experience, accessing the data representing the position of the one or more AR elements of the first AR experience that was stored prior to termination of the first AR experience to generate a display of the second set of AR objects of the second AR experience, the second set of AR objects being placed at a particular position within a second image corresponding to the position of the one or more AR elements of the first set of AR objects of the first AR experience (i.e. when applying AR effects to a previously recorded video, an additional/second AR effect can be displayed where a beginning implementation parameter, e.g. position, for the second AR effect can default to the ending implementation parameter of the first AR effect – Para 58, 97, 98).
Claim 3, Yoon disclose the method of claim 1, wherein the second AR experience is launched by a messaging application implemented on the device (i.e. a telephone computing environment – Para 30 – that uses social graph data – Para 58 – and communicates in a networked environment with other client devices – Para - 40).
Claim 4, Yoon discloses the method of claim 1, further comprising:
receiving a request to terminate the first AR experience, wherein the data is stored in local storage of the device in response to receiving the request to terminate the first AR experience (i.e. determining a condition for ending AR effect selection has occurred – Para 59; store a list of AR effects selected – Para 60; computing device includes storage memory implemented locally – Para 42; Fig. 4).
Claim 5, Yoon discloses the method of claim 1, wherein the first AR experience is resumed on a third device with the remote server, wherein the second image is captured by the third device (i.e. the computing device can act as a server and share a database – Para 40 – with another client device operating in a networked environment with the computing device – Para 38; Fig. 3; the other device having the same system components as the first device can implement the method – Para 42; Fig. 4 – and provide input – Para 24; Fig. 10).
Claim 6, Yoon discloses the method of claim 1, further comprising:
receiving a request to terminate the first AR experience, wherein the data is transmitted to a remote server in response to receiving the request to terminate the first AR experience (i.e. determining a condition for ending AR effect selection has occurred – Para 59; store a list of AR effects selected – Para 60; computing device includes storage memory implemented at a server – Para 42; Fig. 4).
Claim 7, Yoon discloses the method of claim 1, further comprising:
obtaining, from the stored data, an identifier of the real-world object (i.e. obtaining implementation parameter including identification of a depicted object – Para 57; Fig. 4); detecting the real-world object in the second image based on the identifier obtained from the stored data (i.e. recognize an object in the video to initial implementation parameters – Para 57); and positioning the second set of AR elements in the second image based on a placement of the real-world object in the second image (i.e. the display position of a second/subsequent AR effect is at the same display position, e.g. at a mouth, as a previous/first AR effect located on the video image of a person – Fig. 10C; Para 57, 58).
Claim 8, Yoon discloses the method of claim 1, wherein the particular position corresponds to the position of the one or more AR elements relative to the real-world object depicted in the first image (i.e. the display position of a second/subsequent AR effect is at the same display position, e.g. at a mouth, as a previous/first AR effect located on the video image of a person – Fig. 10C; Para 57, 58).
Claim 10, Yoon discloses determining and storing a real object for association with display of AR effects (Para 57, 58; Fig. 1, 3, 4), as does Auten (Para 5, 10, 31).
Auten discloses the method of claim 1, further comprising: determining that the real-world object corresponds to a landmark (i.e. recognizing a real world object – Para 41 – as an object that corresponds to a landmark – Para 5, 10, 31);
storing the landmark as part of the data (i.e. store the landmark in association with the AR elements display position – Para 5, 10, 49); and
causing the one or more AR elements to be displayed in a subsequent presentation of the first AR experience in which the landmark appears (i.e. persistent display of the AR elements/objects in relation to the landmark – Para 66), which Yoon in view of McTaggart fails to disclose.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention at the time invention was made to combine Auten’s known method of determining that the real-world object corresponds to a landmark; storing the landmark as part of the data; and causing the one or more AR elements to be displayed in a subsequent presentation of the first AR experience in which the landmark appears with the method of Yoon in view of McTaggart because each displays AR effects in association with a recognized real object, of which a landmark is exemplary. Thus, the combination yields predictable results.
Claim 13, Yoon discloses the method of claim 1, wherein the data comprises location information of the real-world object, the location information comprising at least one of a latitude and longitude of the real-world object, global positioning system coordinates of the real-world object, a bounding box associated with the real-world object, an object type of the real-world object (i.e. associate location information of AR effect at a feature, e.g. mouth, of the real object – Para 97; Fig. 10), or a three-dimensional match of the real-world object.
Independent claim 15, the claim is similar in scope to claim 1. Therefore, similar rationale as applied in the rejection of claim 1 applies herein.
Claims 16-18, and 20, the corresponding rationale as applied in the rejection of claims 1-6 apply herein.
Independent claim 19, the claim is similar in scope to claim 1. Therefore, similar rationale as applied in the rejection of claim 1 applies herein.
Claim(s) 9 and 11-12, is/are rejected under 35 U.S.C. 103 as being unpatentable over Kyle Yoon et al., US 2021/0407151 A1 and Robert Auten et al., US 2017/0365098 A1, and Ryan McTaggart et al., US 2017/0084082 A1, as applied to claim 1, and further in view of Ryan Fink et al., US 2020/0090409 A1.
Claim 9, Yoon discloses using the stored data to automatically position the second set of AR elements within the second AR experience (i.e. an additional/second AR effect can be displayed where a beginning implementation parameter, e.g. position, for the second AR effect can default to the ending implementation parameter of the first AR effect – Para 58, 97, 98), as does Auten (i.e. AR experiences may be persistent – Para 66), as does McTaggart (i.e. AR experience may be locked – Para 79).
Fink discloses the method of claim 1, further comprising:
determining that the second AR experience is related to the first AR experience (i.e. performing correlation by comparing recognized stored scene objects with objects from a first captured image – Para 40); and
using the stored data to automatically position the second set of AR elements within the second AR experience in response to determining that the second AR experience is related to the first AR experience (i.e. positioning superimposed objects based on the real object, e.g. landmark, alignment in the second captured image – Para 47; AR data, e.g. 3D objects/annotations, are displayed in the position corresponding to the original placement on the first client device – Para 39, 40), which Yoon in view of Auten and McTaggart fail to disclose.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention at the time invention was made to combine Fink’s known method of determining that the second AR experience is related to the first AR; and using the stored data to automatically position the second set of AR elements within the second AR experience in response to determining that the second AR experience is related to the first AR experience with the method of Yoon in view of Auten and McTaggart because each displays AR objects in association with a real object based on a parameter, which provides the benefit of maintaining a display position of AR effects associated with a same real object.
Claim 11, Yoon discloses selecting and storing AR objects (Fig. 4, 5) for display in association with recognized real world objects (Para 58; Fig. 10), as does Auten (abstract), as does McTaggart (Fig. 4B, 5).
Fink discloses the method of claim 1, further comprising:
generating an object classifier based on the real-world object depicted in the first image (i.e. create a fingerprint of a scene including anchor point and identifiable landmarks – Para 45);
storing the object classifier as part of the data (i.e. store fingering data including anchor point and landmarks - Para 45);
obtaining object classifiers for one or more objects that are depicted in the second image (i.e. performing correlation by comparing recognized stored scene objects with objects from a first captured image – Para 40);
determining that the object classifier in the data matches a given one of the object classifiers (i.e. determining that the recognized scene objects matches with stored fingerprint data, e.g. identifiable landmarks/shapes– Para 40, 45); and
causing the one or more AR elements to be displayed in the second image in response to determining that the object classifier in the data matches the given one of the object classifiers (i.e. positioning superimposed objects based on the real object, e.g. landmark, alignment in the second captured image – Para 47; AR data, e.g. 3D objects/annotations, are displayed in the position corresponding to the original placement on the first client device – Para 39, 40), which Yoon in view of Auten and McTaggart fails to disclose.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention at the time invention was made to combine Fink’s known method of generating an object classifier based on the real-world object depicted in the first image; storing the object classifier as part of the data; obtaining object classifiers for one or more objects that are depicted in the second image; determining that the object classifier in the data matches a given one of the object classifiers; and
causing the one or more AR elements to be displayed in the second image in response to determining that the object classifier in the data matches the given one of the object classifiers with the method of Yoon in view of Auten and McTaggart because each displays AR objects in association with a real object based on a parameter, of which an object classification is exemplary. Thus, the combination yields predictable results.
Claim 12, Yoon discloses selecting and storing AR objects (Fig. 4, 5) for display in association with recognized real world objects (Para 58; Fig. 10), as does Auten (abstract), as does McTaggart (Fig. 4B, 5).
Fink discloses the method of claim 11, wherein the one or more AR elements are displayed at a same position in the second image relative to the given one of the object classifiers as the position in the first image (i.e. manipulated AR data, e.g. 3D objects, may be saved and later recreated – Para 48; Fig. 3; a client device displays AR data at the same locations as displayed on the first client device – Para 38), which Yoon in view of Auten and McTaggart fails to disclose.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention at the time invention was made to combine Fink’s known method of wherein the one or more AR elements are displayed at a same position in the second image relative to the given one of the object classifiers as the position in the first image with the method of Yoon in view of Auten and McTaggart because each displays AR objects in association with a real object based on a parameter, of which an object classification is exemplary. Thus, the combination yields predictable results.
Claim(s) 14, is/are rejected under 35 U.S.C. 103 as being unpatentable over Kyle Yoon et al., US 2021/0407151 A1 and Robert Auten et al., US 2017/0365098 A1 and Ryan McTaggart et al., US 2017/0084082 A1, as applied to claim 1, and further in view of Matthew Wren et al., US 2021/0166487 A1.
Claim 14, Yoon discloses the method of claim 1, a graphical user interface for enabling storage of location data for AR elements after termination of the AR experience (i.e. a user interface displaying AR effects – Fig. 10C; process stores the implementation parameters including the position of the AR effects, after ending display of the AR effects as applied to a live video – Para 59, 60), as does Auten (i.e. AR experiences may be persistent – Para 66), as does McTaggart (i.e. storing changes to an AR experience – Para 82).
Yoon in view of Auten and McTaggart fails to disclose the method of claim 1, wherein the AR experience is developed using an AR developer platform, the AR developer platform comprising a graphical user interface for enabling storage of location data for AR elements, which Wren discloses (i.e. a content management system used in creating an AR experience allows users to manage and store digital content – Para 4; end user interaction with displayed assets is stored – Para 68; and analytic information of the end user interaction is stored – Para 44, 47).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention at the time invention was made to combine Wren’s AR developer platform comprising a graphical user interface for enabling storage of location data for AR elements with the method of Yoon in view of Auten and McTaggart because the developer platform enables installment of the application at differing client devices and client device access to the same functions and display elements for the benefit of providing persistent display of augmentation among the differing devices.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHANTE HARRISON whose telephone number is (571)272-7659. The examiner can normally be reached Monday - Friday 8:00 am to 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHANTE E HARRISON/Primary Examiner, Art Unit 2615
Prosecution Insights