DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1, 7, 8, 18-19 are amended.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 04/28/2026 is being considered by the examiner. The submission is in compliance with the provisions of 37 CFR 1.97.
Drawings
The drawings were received on 06/04/2024. These drawings are acceptable. Previous Drawings Objections is withdrawn.
Specification
35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, requires the specification to be written in “full, clear, concise, and exact terms.” The specification is replete with terms which are not clear, concise and exact. The specification should be revised carefully in order to comply with 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112. Examples of some unclear, inexact or verbose terms used in the specification are: Many important terms that are cited in Drawings and in the Detailed Description a couple of times, are not cited throughout the rest of paragraphs in the Detailed Description. In paragraph 0163, it discloses Fig. 1A and labels “scene 105” with reference number 105 once. Then, paragraph 0164 mentions “scene” four times, however does not properly label “scene” in paragraph 0164. As well as many other paragraphs thought the Detailed Description without a proper label for “scene”. Another example is Paragraph 0029, which labels “display generation component 120” with reference number 120 twice. Then mentions “display generation component” multiple times throughout the detailed disclosure without a proper referencing numbering. As well as many other paragraphs without a proper referencing numbering for “display generation component”, like paragraph 0039. This goes for many different terms throughout the detailed disclosure that do not have a proper label reference. Many inconsistent reference numberings for terms that were previously numbered. The examples given are a non-exhaustive list, this applies to many different terms throughout the detailed disclosure that have inconsistent reference numbering. Further revision is required, additional inconsistence should be fixed accordingly.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 2, 9, 17, 18 and 19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by MALIA (No. CN-114402275-A “Malia”).
Regarding claim 1, Malia teaches “(Currently Amended) A method comprising: at a computer system in communication with a display generation component and one or more input devices:” (According to some embodiments, the method is performed on a computer system having a display generation component and one or more input devices; Page 2, Para 4);
“displaying, via the display generation component, a user interface” (displaying a user interface; Page 2, Para 3); “in a three- dimensional environment from a viewpoint of a user of the computer system, wherein the user interface” (user interface objects may be displayed in the displayed augmented reality environment... the computer system (eg, the pose of a display device used to display a user interface to a user of the computer system) middle.; Page 14, Para 1);
Malia discloses a displayed augmented reality environment used to display the user interface to a user which teaches the claimed subject matter of the 3D environment viewpoint of a user of the computer system.
“includes a first representation of a first location experience” (a representation of the physical environment, wherein the representation of the physical environment includes a representation of a first physical object occupying a first physical space in the physical environment and having a first corresponding object attribute; Page 2, Para 4); “and a first portion of a navigation user interface element with a first orientation” (12E-12F illustrate the process by which a user performs a zoom-in operation on a still view representation of the physical environment 1200; Page 55, Para 9); “that includes a first representation of a first point of interest experience associated with the first location experience” (In response, FIG. 5I shows displaying a real-time representation 518 of the content displayed in the room 502 based on the location of the user 501; Page 28, Para 1);
“while displaying, via the display generation component, the user interface from the viewpoint of the user of the computer system:” (FIG. 5A also shows the display generation component 308 of the computer system 301 - b that the user 501 is currently looking at when scanning the room 502. Display 308 shows a user interface 517 that shows a live representation 518 (sometimes referred to herein as a live view representation) of what the camera 305 is currently capturing; Page 26, Para 2);
Malia discloses the user looking at a live representation or live view of what is being captured by the display generation component of the computer system. This teaches the claimed subject matter of the user interface from the viewpoint of the user of the computer.
detecting, via the one or more input devices, a first input directed to the first representation of the first location experience; and” (The method includes detecting a first input corresponding to a virtual object, wherein movement of the first input corresponds to a request to move the virtual object in the representation of the physical environment relative to the representation of the first physical object; Page 2, Para 4);
“in response to detecting the first input:
changing the display of the navigation user interface element from the first orientation to a second orientation corresponding to the first location experience without changing the viewpoint of the user of the computer system; and” (a virtual object at a location in the representation of the physical environment that corresponds to a second physical space in the physical environment that is different from the first physical space; Page 2, Para 4); (the placement user interface element displayed at a predefined location relative to the display generating component (eg, a predefined The location is fixed regardless of which parts of the physical environment were captured in the previously captured representation of the media item).... performing one or more transformations (eg, zooming in, zooming out, translating, rotating, and/or converting a representation of the first previously captured media item) which performs other transformations of the input), the system generates via the display while maintaining the display of the placement user interface element relative to a predefined location of the display generating component (e.g. keeping the placement indicator stationary relative to the display generating component); Page 72, Para 4);
Malia teaches a location the user is viewing is fixed while they are able to preform transformations like panning, zoom, etc. Thus, with a fixed viewpoint, the orientation location can transform without changing to a new viewpoint. This teaches the claimed subject matter of the viewpoint of the user not changing while allowing for the orientations to transform based on one or more transformations.
“displaying the navigation user interface element to include a second portion, different from the first portion and associated with the first location experience.” (performed on a computer system having a display generation component and one or more input devices.... after receiving the input, displaying an annotation on a portion of the displayed second representation of the second previously captured media, wherein the second previously captured media is different from the first previously captured media and the second representation is; Page 3, Para 1);
Regarding claim 2, Malia teaches “(Original) The method of claim 1, further comprising: while displaying the navigation user interface element: displaying, via the display generation component, a second representation of the first location experience that includes animated content;” (displaying an animated transition from the representation of the first previously captured media item to the representation of the second previously captured media item; Page 3, Para 4);
“receiving, via the one or more input devices, an input selecting the second representation of the first location experience; and” (after (eg, in response to) receiving (1012) an input corresponding to a request to annotate the portion of the first representation, and on the portion of the displayed second representation of the second media… first media (eg, in response to an input corresponding to a selection of the second media) to display in the first representation of the first media; Page 47, Para 7);
“in response to receiving the input selecting the second representation of the first location experience, displaying, via the display generation component, the first location experience.” ((eg, in response to an input corresponding to a selection of the second media) to a physical environment represented in the first representation of the first media based on the determination; Page 48, Para 5);
Regarding claim 9, Malia teaches “(Original) The method of claim 1, further comprising: while displaying the first location experience and the navigation user interface element: displaying, via the display generation component, the first location experience including video having a respective viewpoint that changes over time; and” (one or more cameras (eg, continuously or in fixed A camera that provides a real-time preview of at least a portion of the content within the camera's field of view at intervals and optionally generates a video output including one or more image frames capturing the content within the camera's field of view); Page 4, Para 2);
“displaying, via the display generation component, a visual indication of the respective viewpoint at a location relative to the navigation user interface element that changes over time in accordance with the respective viewpoint that changes over time.” (The method includes when displaying the representation of the field of view: updating the representation of the field of view over time based on changes in the field of view. The change in the field of view includes the movement of the first entity moving the first anchor point, and as the first anchor point moves along the path in the physical environment, the corresponding portion of the representation of the first entity corresponding to the first anchor point moves along the path in the physical environment; Page 4, Para 1);
Regarding claim 17, Malia teaches “(Original) The method of claim 1, further comprising: while displaying, via the display generation component, the first location experience, receiving, via the one or more input devices, one or more inputs corresponding to a request to add an annotation to the first location experience; and” (Upon displaying the first representation of the first media, receiving (1006) corresponds to annotating a portion of the first representation corresponding to the first portion of the physical environment (eg, by adding a virtual object or modifying an existing displayed virtual object, such as 6D-6E, where a note 606 is added to the input of the request to expand the media item 604); Page 47, Para 4);
“in response to receiving the one or more inputs, modifying the first location experience to include the annotation.” (In response to receiving the input, displaying an annotation on the portion of the first representation corresponding to the first portion of the physical environment, the annotation having an annotation based on the physical environment; Page 47, Para 5);
Regarding claim 18, Malia teaches “(Currently Amended) A computer system that is in communication with a display generation component and one or more input devices, the computer system comprising:” (According to some embodiments, a computer system (eg, electronic device) includes display generating components (eg, display, projector, head-mounted display, head-up display, etc.), one or more cameras … and one or more input devices; Page 4, Para 2);
“one or more processors;” (one or more processors; Page 2, Para 1);
“memory; and” (memory; Page 2, Para 1);
“one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:” (one or more modules, programs or sets of instructions stored in the memory for performing various functions; Page 2, Para 1);
“displaying, via the display generation component, a user interface” (displaying a user interface; Page 2, Para 3); “in a three- dimensional environment from a viewpoint of a user of the computer system, wherein the user interface” (user interface objects may be displayed in the displayed augmented reality environment... the computer system (eg, the pose of a display device used to display a user interface to a user of the computer system) middle.; Page 14, Para 1);
Malia discloses a displayed augmented reality environment used to display the user interface to a user which teaches the claimed subject matter of the 3D environment viewpoint of a user of the computer system.
“includes a first representation of a first location experience” (a representation of the physical environment, wherein the representation of the physical environment includes a representation of a first physical object occupying a first physical space in the physical environment and having a first corresponding object attribute; Page 2, Para 4); “and a first portion of a navigation user interface element with a first orientation” (12E-12F illustrate the process by which a user performs a zoom-in operation on a still view representation of the physical environment 1200; Page 55, Para 9); “that includes a first representation of a first point of interest experience associated with the first location experience” (In response, FIG. 5I shows displaying a real-time representation 518 of the content displayed in the room 502 based on the location of the user 501; Page 28, Para 1);
“while displaying, via the display generation component, the user interface from the viewpoint of the user of the computer system:” (FIG. 5A also shows the display generation component 308 of the computer system 301 - b that the user 501 is currently looking at when scanning the room 502 . Display 308 shows a user interface 517 that shows a live representation 518 (sometimes referred to herein as a live view representation) of what the camera 305 is currently capturing; Page 26, Para 2);
Malia discloses the user looking at a live representation or live view of what is being captured by the display generation component of the computer system. This teaches the claimed subject matter of the user interface from the viewpoint of the user of the computer.
detecting, via the one or more input devices, a first input directed to the first representation of the first location experience; and” (The method includes detecting a first input corresponding to a virtual object, wherein movement of the first input corresponds to a request to move the virtual object in the representation of the physical environment relative to the representation of the first physical object; Page 2, Para 4);
“in response to detecting the first input:
changing the display of the navigation user interface element from the first orientation to a second orientation corresponding to the first location experience without changing the viewpoint of the user of the computer system; and” (a virtual object at a location in the representation of the physical environment that corresponds to a second physical space in the physical environment that is different from the first physical space; Page 2, Para 4); (the placement user interface element displayed at a predefined location relative to the display generating component (eg, a predefined The location is fixed regardless of which parts of the physical environment were captured in the previously captured representation of the media item).... performing one or more transformations (eg, zooming in, zooming out, translating, rotating, and/or converting a representation of the first previously captured media item) which performs other transformations of the input), the system generates via the display while maintaining the display of the placement user interface element relative to a predefined location of the display generating component (e.g. keeping the placement indicator stationary relative to the display generating component); Page 72, Para 4);
Malia teaches a location the user is viewing is fixed while they are able to preform transformations like panning, zoom, etc. Thus, with a fixed viewpoint, the orientation location can transform without changing to a new viewpoint. This teaches the claimed subject matter of the viewpoint of the user not changing while allowing for the orientations to transform based on one or more transformations.
“displaying the navigation user interface element to include a second portion, different from the first portion and associated with the first location experience.” (performed on a computer system having a display generation component and one or more input devices.... after receiving the input, displaying an annotation on a portion of the displayed second representation of the second previously captured media, wherein the second previously captured media is different from the first previously captured media and the second representation is; Page 3, Para 1);
Regarding claim 19, Malia teaches “(Currently Amended) A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of a computer system that is in communication with a display generation component and one or more input devices, cause the computer system to perform a method comprising:” (According to some embodiments, a computer-readable storage medium has stored therein instructions that, when executed by a computer system comprising (and/or in communication with) a display generation component, one or more cameras, one or more input devices, optionally one or more gesture sensors, optionally one or more sensors for detecting intensity of contacts with a touch-sensitive surface, and optionally one or more tactile output generators, cause the computer system to perform, or cause the performance of, the operations of any of the methods described herein; Page 4, Para 2);
“displaying, via the display generation component, a user interface” (displaying a user interface; Page 2, Para 3); “in a three-dimensional environment from a viewpoint of a user of the computer system, wherein the user interface” (user interface objects may be displayed in the displayed augmented reality environment... the computer system (eg, the pose of a display device used to display a user interface to a user of the computer system) middle.; Page 14, Para 1);
Malia discloses a displayed augmented reality environment used to display the user interface to a user which teaches the claimed subject matter of the 3D environment viewpoint of a user of the computer system.
“includes a first representation of a first location experience” (a representation of the physical environment, wherein the representation of the physical environment includes a representation of a first physical object occupying a first physical space in the physical environment and having a first corresponding object attribute; Page 2, Para 4); “and a first portion of a navigation user interface element with a first orientation” (12E-12F illustrate the process by which a user performs a zoom-in operation on a still view representation of the physical environment 1200; Page 55, Para 9); “that includes a first representation of a first point of interest experience associated with the first location experience” (In response, FIG. 5I shows displaying a real-time representation 518 of the content displayed in the room 502 based on the location of the user 501; Page 28, Para 1);
“while displaying, via the display generation component, the user interface from the viewpoint of the user of the computer system:” (FIG. 5A also shows the display generation component 308 of the computer system 301 - b that the user 501 is currently looking at when scanning the room 502 . Display 308 shows a user interface 517 that shows a live representation 518 (sometimes referred to herein as a live view representation) of what the camera 305 is currently capturing; Page 26, Para 2);
Malia discloses the user looking at a live representation or live view of what is being captured by the display generation component of the computer system. This teaches the claimed subject matter of the user interface from the viewpoint of the user of the computer.
detecting, via the one or more input devices, a first input directed to the first representation of the first location experience; and” (The method includes detecting a first input corresponding to a virtual object, wherein movement of the first input corresponds to a request to move the virtual object in the representation of the physical environment relative to the representation of the first physical object; Page 2, Para 4);
“in response to detecting the first input:
changing the display of the navigation user interface element from the first orientation to a second orientation corresponding to the first location experience without changing the viewpoint of the user of the computer system; and” (a virtual object at a location in the representation of the physical environment that corresponds to a second physical space in the physical environment that is different from the first physical space; Page 2, Para 4); (the placement user interface element displayed at a predefined location relative to the display generating component (eg, a predefined The location is fixed regardless of which parts of the physical environment were captured in the previously captured representation of the media item).... performing one or more transformations (eg, zooming in, zooming out, translating, rotating, and/or converting a representation of the first previously captured media item) which performs other transformations of the input), the system generates via the display while maintaining the display of the placement user interface element relative to a predefined location of the display generating component (e.g. keeping the placement indicator stationary relative to the display generating component); Page 72, Para 4);
Malia teaches a location the user is viewing is fixed while they are able to perform transformations like panning, zoom, etc. Thus, with a fixed viewpoint, the orientation location can transform without changing to a new viewpoint. This teaches the claimed subject matter of the viewpoint of the user not changing while allowing for the orientations to transform based on one or more transformations.
“displaying the navigation user interface element to include a second portion, different from the first portion and associated with the first location experience.” (performed on a computer system having a display generation component and one or more input devices.... after receiving the input, displaying an annotation on a portion of the displayed second representation of the second previously captured media, wherein the second previously captured media is different from the first previously captured media and the second representation is; Page 3, Para 1);
Allowable Subject Matter
Claims 3-8, and 10-16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Arguments
Applicant’s arguments, see pg. 27, filed 06/04/2024, with respect to Drawings have been fully considered and are persuasive. The Objection of the Drawings has been withdrawn.
Applicant’s arguments, see pg. 29, filed 04/28/2026, with respect to Claim 7 have been fully considered and are persuasive. The Rejection of Claim 7 has been withdrawn.
The Applicant has amended claim 7. Claim 7 112b rejection is rendered moot by present amendment of claim 7.
The Examiner replies that the 112b rejection of claim 5 is withdrawn due to the amendment of claim 7.
Applicant's arguments filed 04/28/2026 have been fully considered but they are not persuasive. Regarding the 35 USC 102(a)(1) rejection of claims 1, 18 and 19, The Applicant argues the characterization of Malia fails to disclose, teach or suggest the amendments to the independent claims 1, 18 and 19. The Applicant argues that Malia fails to teach "displaying, via the display generation component, a user interface in a three-dimensional environment from a viewpoint of the user of the computer system, wherein the user interface includes a first representation of a first location experience and a first portion of a navigation user interface element with a first orientation that includes a first representation of a first point of interest experience associated with the first location experience"; “while displaying, via the display generation component, the user interface from the viewpoint of the user of the computer system:” and in response to detecting a first input directed to the first representation of the first location experience, "changing the display of the navigation user interface element from the first orientation to a second orientation corresponding to the first location experience without changing the viewpoint of the user of the computer system."
The Examiner replies that Malia anticipates the applicant’s claim language.
“displaying, via the display generation component, a user interface” (displaying a user interface; Page 2, Para 3); “in a three- dimensional environment from a viewpoint of a user of the computer system, wherein the user interface” (user interface objects may be displayed in the displayed augmented reality environment... the computer system (eg, the pose of a display device used to display a user interface to a user of the computer system) middle.; Page 14, Para 1);
Malia discloses a displayed augmented reality environment used to display the user interface to a user which teaches the claimed subject matter of the 3D environment viewpoint of a user of the computer system.
“while displaying, via the display generation component, the user interface from the viewpoint of the user of the computer system:” (FIG. 5A also shows the display generation component 308 of the computer system 301 - b that the user 501 is currently looking at when scanning the room 502 . Display 308 shows a user interface 517 that shows a live representation 518 (sometimes referred to herein as a live view representation) of what the camera 305 is currently capturing; Page 26, Para 2);
Malia discloses the user looking at a live representation or live view of what is being captured by the display generation component of the computer system. This teaches the claimed subject matter of the user interface from the viewpoint of the user of the computer.
“in response to detecting the first input:
changing the display of the navigation user interface element from the first orientation to a second orientation corresponding to the first location experience without changing the viewpoint of the user of the computer system; and” (a virtual object at a location in the representation of the physical environment that corresponds to a second physical space in the physical environment that is different from the first physical space; Page 2, Para 4); (the placement user interface element displayed at a predefined location relative to the display generating component (eg, a predefined The location is fixed regardless of which parts of the physical environment were captured in the previously captured representation of the media item).... performing one or more transformations (eg, zooming in, zooming out, translating, rotating, and/or converting a representation of the first previously captured media item) which performs other transformations of the input), the system generates via the display while maintaining the display of the placement user interface element relative to a predefined location of the display generating component (e.g. keeping the placement indicator stationary relative to the display generating component); Page 72, Para 4);
Malia teaches a location the user is viewing is fixed while they are able to perform transformations like panning, zoom, etc. Thus, with a fixed viewpoint, the orientation location can transform without changing to a new viewpoint. This teaches the claimed subject matter of the viewpoint of the user not changing while allowing for the orientations to transform based on one or more transformations.
The Applicant argues that Malia fails to establish that the degree in which an annotation is visible in a media representation of Malia is analogous to the claim limitations "displaying the navigation user interface element to include a second portion, different from the first portion and associated with the first location experience in response to detecting the first input directed to the first representation of the first location experience," required by the independent claims, because the partial display of an annotation because the annotation was not captured/outside the capture frame is not the same as a computer system deliberately (in response to detecting the first input directed to the first representation of the first location experience) displaying a second portion associated with the first location experience.
The Examiner replies that Malia teaches the applicant’s claim language.
“displaying the navigation user interface element to include a second portion, different from the first portion and associated with the first location experience.” (performed on a computer system having a display generation component and one or more input devices.... after receiving the input, displaying an annotation on a portion of the displayed second representation of the second previously captured media, wherein the second previously captured media is different from the first previously captured media and the second representation is; Page 3, Para 1);
Malia discloses performing on a computer system, and displaying an annotation that is captured from an input. Which teaches the computer system deliberately (in response to detecting the first input directed to the first representation of the first location experience) displaying a second portion associated with the first location experience.
Therefore, Malia anticipates the claimed subject matter as Malia discloses the viewpoint of the user in a computer system, and without changing the viewpoint of a user in a computer system. In addition, Malia teaches the computer system deliberately displaying a second portion associated with the first location experience. Thus, claims 1, 18, and 19 102(a)(1) rejections are maintained. Due to the maintained rejection of independent claims 1, 18, and 19, claims 3-8 and 10-16 objections are maintained by virtue of dependency on the rejected base claim.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIGITER D PROTAZI whose telephone number is (571)272-7995. The examiner can normally be reached Monday - Friday 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said A Broome can be reached at 5712722931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.D.P./Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights