Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Application
This final office action is in response to the amendment filed 10/14/2025. Claims 1, 3-6, and 11-20 have been amended. Claims 1-20 are currently pending and have been examined below.
Claim Objections
Claim 4 is objected to because of the following informalities: “is based” in line 2 should be replaced with “are based.” Appropriate correction is required.
Claim 5 is objected to because of the following informalities: “wherein the method further comprising the step of” in lines 1-2 should be replaced with “the method further comprises the step of.” Appropriate correction is required.
Claim 6 is objected to because of the following informalities: “wherein the method further comprising the step of” in lines 1-2 should be replaced with “the method further comprises the step of.” Appropriate correction is required.
Claim 14 is objected to because of the following informalities: “wherein the system further comprises a user interface includes” in line 2 should be replaced with “wherein the system further comprises a user interface including.” Appropriate correction is required.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 7-10 and 11-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 7 recites the limitation “the system” in line 1. There is insufficient antecedent basis for this limitation in the claim. Claims 8-10 are rejected by virtue of their dependency on claim 7.
Claim 11 recites the limitation “the user’s device” in line 10. There is insufficient antecedent basis for this limitation in the claim. Claims 12-20 are rejected by virtue of their dependency on claim 11.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 11, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication 20250139838 (“Couleaud”) in view of US Patent Application Publication Number 20220198701 (“Melnik”).
Claim 1
As per claim 1, Couleaud teaches a method for displaying Augmented Reality on a user's
device, comprising the steps of:
capturing image data of a physical environment using an image sensor integrated with the user's device ([0014] “the object of interest, such as, for example, a roadside billboard, is identified and converted for display on a display device.” And, [0137] “image of the billboard previously captured by the front-facing camera.” And, [0152] “information other than billboards is detected.” And, [0158] “detecting a presence of a billboard in the first video stream. The process includes generating a first image of the billboard from the first video stream. The process includes receiving a second video stream from a rear-facing camera in the vehicle. The process includes detecting the billboard in the second video stream.” And, [0012] “the objects of interest are, in particular, roadside billboards. The objects of interest are not limited to billboards and include without limitation any physical conveyance of information.”);
processing the captured image data configured to identify physical objects, including advertising boards and art pieces ([0014] “the object of interest, such as, for example, a roadside billboard, is identified and onverted for display on a display device.” And, [0137] “image of the billboard previously captured by the front-facing camera.” And, [0152] “information other than billboards is detected.” And, [0158] “detecting a presence of a billboard in the first video stream. The process includes generating a first image of the billboard from the first video stream. The process includes receiving a second video stream from a rear-facing camera in the vehicle. The process includes detecting the billboard in the second video stream.” And, [0012] “the objects of interest are, in particular, roadside billboards. The objects of interest are not limited to billboards and include without limitation any physical conveyance of information.”);
generating augmented reality content overlays ([0133] “The SRVMD is configured to provide augmented reality functions, such as overlay information related to a billboard passed by the vehicle.” And, [0106] “The XR environment is an AR or MR environment provided using an AR or MR device.” And, [0129] “acquisition and display of imagery and information pertaining, for example, to roadside information, such as billboards, street name signs, and safety-related objects or information.”);
detecting user input gestures through a touch-sensitive display and gesture recognition hardware, and interpreting these inputs to modify the displayed AR content dynamically; wherein the steps of capturing, processing, rendering, and user input detection are performed with low latency to provide a seamless and interactive augmented reality experience ([0195] “For example, computing device may receive a user input such as a user swipe or user touch.” And, [0196] “User input may be received from a user selection-capturing interface that is separate from device, such as a remote-control device, trackpad, or any other suitable user movement-sensitive, audio-sensitive or capture devices, or as part of device, such as a touchscreen of display.” And, [0158]. And, [0133] “The SRVMD is configured to provide augmented reality functions, such as overlay information related to a billboard passed by the vehicle.” And, [0141] “in response to the user selection of the user-selectable icon (as seen in FIG. 14), the SRVMD displays supplemental content on a right side of the SRVMD.” And, [0041] “The image is configurable in any suitable manner including inversion, magnification, replacement with a supplemental image.” And, [0087] “a consistent direction along an angle with respect to an object (e.g., the direction of gaze movement relative to an edge of the window opening) swept by the gaze, and/or the duration in time of the gaze (e.g., on the order of seconds or fractions of a second).” And, [0165] “identifying in real time or in near real time a safety-related object in the acquired media.” And, [0202] “reduce processing time or enhance processing capabilities for each step in the processes disclosed herein by one skilled in the art.”).
Couleaud teaches processing captured image data to identify physical objects including advertising boards and art pieces but does not explicitly teach the following feature taught by Melnik:
processing the captured image data in real-time via a dedicated image recognition processor configured to identify physical objects, by comparing the image data against a stored database of trained object signatures ([0004] “real-time or near real-time detection of such objects and regions within images and subsequent display of graphical presentation data.” And, [0027] “a digital signal processor (DSP).” And, [0004] “utilize digital signatures to efficiently identify objects of interest within images and cause graphical presentation data to be accurately displayed in association therewith.” And, [0056] “digital signatures stored in the database.” And, [0047] “match a stored digital signature in an instance in which the determined digital signature identically matches the stored digital signature.” And, [0068] “train the query processing module to utilize the digital signature to identify the object of interest within the image captured by an image capture device.”).
Couleaud teaches generating augmented reality content overlays but does not explicitly teach the following feature taught by Melnik:
content overlays precisely aligned with the recognized physical objects using a graphics processing unit (GPU)-based rendering engine that synchronizes rendering with device orientation and spatial position sensors ([0060] “graphical assets may be overlaid on one or more graphical data placement regions such that additional information is added to the augmented reality environment for viewing.” And, [0007] “image capture device comprising at least one of a position or orientation of the image capture device.” And, [0048] “the graphical assets that are received may be further based on camera position and/or orientation included in the graphical data placement information. For example, in a scenario in which a graphical asset of an arrow is to be overlaid on or near a building such that the arrow points to the building in order to direct a person to the building, the arrow graphical asset that is received may depend on what direction the user is approaching the building from. For example, if the building is ahead and to the right of the user, a right-facing arrow may be received.” And, [0005] “causing display comprises overlaying at least one graphical asset of the one or more graphical assets on the at least one graphical data placement region.” And, [0008] “when executed by the processing circuity, cause the apparatus to receive graphical presentation data comprising one or more graphical assets.” And, [0027] “a hardware accelerator, a special-purpose computer chip.”).
Therefore, it would have been obvious to modify Couleaud to include processing the captured image data in real-time via a dedicated image recognition processor configured to identify physical objects, by comparing the image data against a stored database of trained object signatures and content overlays precisely aligned with the recognized physical objects using a graphics processing unit (GPU)-based rendering engine that synchronizes rendering with device orientation and spatial position sensors as taught by Melnik in order to utilize digital signatures to efficiently identify objects of interest within images and cause graphical presentation data to be accurately displayed in association therewith (Melnik [0004]).
Claim 2
As per claim 2, Couleaud further teaches:
wherein the Augmented Reality content includes but not limited to digital overlays, animations, videos, and interactive elements associated with the identified objects ([0133] “The SRVMD is configured to provide augmented reality functions, such as overlay information related to a billboard passed by the vehicle.” And, [0106] “The XR environment is an AR or MR environment provided using an AR or MR device.” And, [0129] “acquisition and display of imagery and information pertaining, for example, to roadside information, such as billboards, street name signs, and safety-related objects or information.” And, [0108] “XR content further includes metadata indicating a relative location, movement data and/or timing data for an XR object, animation and/or scene.” And, [0158] “generating the first image of the billboard from the first video stream comprises extracting the image of the billboard from the video stream using image segmentation or the like in some embodiments. The generating the first image of the billboard from the first video stream comprises receiving an image from a billboard management service to provide personalized overlay for the billboard.”).
Claim 11
As per claim 11, Couleaud teaches a system for displaying Augmented Reality (AR) content,
comprising:
a hardware-based image recognition module including an image sensor configured to capture image data of physical objects ([0014] “the object of interest, such as, for example, a roadside billboard, is identified and converted for display on a display device.” And, [0137] “image of the billboard previously captured by the front-facing camera.” And, [0152] “information other than billboards is detected.” And, [0158] “detecting a presence of a billboard in the first video stream. The process includes generating a first image of the billboard from the first video stream. The process includes receiving a second video stream from a rear-facing camera in the vehicle. The process includes detecting the billboard in the second video stream.” And, [0012] “the objects of interest are, in particular, roadside billboards. The objects of interest are not limited to billboards and include without limitation any physical conveyance of information.”);
generate AR content overlays ([0133] “The SRVMD is configured to provide augmented reality functions, such as overlay information related to a billboard passed by the vehicle.” And, [0106] “The XR environment is an AR or MR environment provided using an AR or MR device.” And, [0129] “acquisition and display of imagery and information pertaining, for example, to roadside information, such as billboards, street name signs, and safety-related objects or information.”);
a tactile interface module comprising a touch-sensitive display and gesture recognition hardware configured to detect user input gestures and enable interactive engagement with the AR content displayed, wherein the tactile interface module transmits the detected inputs to the rendering engine for real-time content updates; wherein the hardware-based image recognition module, AR rendering engine, and tactile interface module are integrated to provide a real-time, responsive AR experience on the user's device ([0195] “For example, computing device may receive a user input such as a user swipe or user touch.” And, [0196] “User input may be received from a user selection-capturing interface that is separate from device, such as a remote-control device, trackpad, or any other suitable user movement-sensitive, audio-sensitive or capture devices, or as part of device, such as a touchscreen of display.” And, [0158]. And, [0133] “The SRVMD is configured to provide augmented reality functions, such as overlay information related to a billboard passed by the vehicle.” And, [0141] “in response to the user selection of the user-selectable icon (as seen in FIG. 14), the SRVMD displays supplemental content on a right side of the SRVMD.” And, [0041] “The image is configurable in any suitable manner including inversion, magnification, replacement with a supplemental image.” And, [0087] “a consistent direction along an angle with respect to an object (e.g., the direction of gaze movement relative to an edge of the window opening) swept by the gaze, and/or the duration in time of the gaze (e.g., on the order of seconds or fractions of a second).” And, [0165] “identifying in real time or in near real time a safety-related object in the acquired media.” And, [0202] “reduce processing time or enhance processing capabilities for each step in the processes disclosed herein by one skilled in the art.”).
Couleaud teaches processing captured image data to identify physical objects including advertising boards and art pieces but does not explicitly teach the following feature taught by Melnik:
a dedicated processing unit configured to identify the physical objects in real-time by analysing the captured image data using a trained machine learning model stored in nontransitory memory ([0004] “real-time or near real-time detection of such objects and regions within images and subsequent display of graphical presentation data.” And, [0024] “a neural network that has been trained as described below to recognize the object of interest utilizing the respective digital signature.” And, [0027] “a digital signal processor (DSP).” And, [0074] “the signature encoding module may comprise a plurality of neural network layers configured to decompose the reference image.” And, [0004] “utilize digital signatures to efficiently identify objects of interest within images and cause graphical presentation data to be accurately displayed in association therewith.” And, [0056] “digital signatures stored in the database.” And, [0047] “match a stored digital signature in an instance in which the determined digital signature identically matches the stored digital signature.” And, [0068] “train the query processing module to utilize the digital signature to identify the object of interest within the image captured by an image capture device.”).
Couleaud teaches generating augmented reality content overlays but does not explicitly teach the following feature taught by Melnik:
a dedicated AR rendering engine implemented on a graphics processing unit (GPU) configured to generate AR content overlays aligned precisely with the recognized physical objects, based on the user's device spatial orientation and position, wherein the rendering engine operates with low latency to maintain seamless AR visualization ([0060] “graphical assets may be overlaid on one or more graphical data placement regions such that additional information is added to the augmented reality environment for viewing.” And, [0007] “image capture device comprising at least one of a position or orientation of the image capture device.” And, [0048] “the graphical assets that are received may be further based on camera position and/or orientation included in the graphical data placement information. For example, in a scenario in which a graphical asset of an arrow is to be overlaid on or near a building such that the arrow points to the building in order to direct a person to the building, the arrow graphical asset that is received may depend on what direction the user is approaching the building from. For example, if the building is ahead and to the right of the user, a right-facing arrow may be received.” And, [0005] “causing display comprises overlaying at least one graphical asset of the one or more graphical assets on the at least one graphical data placement region.” And, [0008] “when executed by the processing circuity, cause the apparatus to receive graphical presentation data comprising one or more graphical assets.” And, [0027] “a hardware accelerator, a special-purpose computer chip.” And, [0082] “identified in received images captured by an image capture device and graphical presentation data may be subsequently displayed in or in near real-time and in a manner that conserves processing resources and is more timely than at least some other image analysis techniques.”).
Therefore, it would have been obvious to modify Couleaud to include a dedicated processing unit configured to identify the physical objects in real-time by analysing the captured image data using a trained machine learning model stored in nontransitory memory and a dedicated AR rendering engine implemented on a graphics processing unit (GPU) configured to generate AR content overlays aligned precisely with the recognized physical objects, based on the user's device spatial orientation and position, wherein the rendering engine operates with low latency to maintain seamless AR visualization as taught by Melnik in order to utilize digital signatures to efficiently identify objects of interest within images and cause graphical presentation data to be accurately displayed in association therewith (Melnik [0004]).
Claim 14
As per claim 14, Couleaud further teaches:
wherein the system further comprises a user interface includes intuitive controls and navigation options for seamless AR content exploration ([0091] “the user is provided options to perform media controls.” [[0195] “For example, computing device may receive a user input such as a user swipe or user touch.” And, [0196] “User input may be received from a user selection-capturing interface that is separate from device, such as a remote-control device, trackpad, or any other suitable user movement-sensitive, audio-sensitive or capture devices, or as part of device, such as a touchscreen of display.” And, [0158] “ And, [0133] “The SRVMD is configured to provide augmented reality functions, such as overlay information related to a billboard passed by the vehicle. Interactive functionalities are provided to re-enforce the advertising message on the billboard.”).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3-6 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication 20250139838 (“Couleaud”) in view of US Patent Application Publication Number 20220198701 (“Melnik”) as applied to claims 1 and 14 above, and in view of US Patent Application Publication Number 20200258099 (“Alghamdi”).
Claim 3
As per claim 3, Couleaud does not explicitly teach but Alghamdi teaches:
wherein capturing Augmented Reality experiences and user interactions comprises dynamically sourcing real time data from surroundings encountered during a taxi ride, processing said data to generate interactive augmented reality content and presenting said content on a user's mobile device in real time, including but not limited to advertisements, informational overlays and contextual prompts ([0100] “The AR overlay will display on the phone screen to direct and guide the other peers with audible prompts from, for example, an avatar.” And, [0017] “the content could comprise an augmented reality vehicle that leads users to interactive experiences.” And, [0022] “interacting with the message in AR or virtual reality (VR) form, playing interactive games, participating in the experiences and messages and sharing them on social media or other platforms.” And, [0079] “The first message is an interactive advertisement.” And, [0091] “receives a real time notification of the current location of the virtual gift from peer.” And, [0021] “provided with real-time data that allows the sender to understand the effectiveness of a message and its targeting.”).
Therefore, it would have been obvious to modify the combination of Couleaud and Melnik to include wherein capturing Augmented Reality experiences and user interactions comprises dynamically sourcing real time data from surroundings encountered during the taxi ride, processing said data to generate interactive augmented reality content and presenting said content on a user's mobile device in real time, including but not limited to advertisements, informational overlays and contextual prompts as taught by Alghamdi in order “to improve engagement” (Alghamdi [0023]) with the augmented reality content in Couleaud.
Claim 4
As per claim 4, Couleaud further teaches:
wherein the Augmented Reality experiences is based on user preferences, geographic location, and interaction history during the taxi ride ([0012] “an interest in the identified object is determined. In another example, one or more objects are identified based on at least one of the user's gaze, a position of one or more of the user's eyes, a direction of the user's gaze, a duration of time that the user's gaze is incident on an object, a speed of movement of the user's gaze, a determined interest of another passenger in the vehicle, a determined interest of one or more other people that have passed through the same location, a trained model, one or more user preferences.” And, [0133] “The SRVMD is configured to provide augmented reality functions, such as overlay information related to a billboard passed by the vehicle.”)
Claim 5
As per claim 5, Couleaud does not explicitly teach but Alghamdi teaches:
incentivizing user engagement with augmented reality content by rewarding users with points, vouchers, or redeemable rewards based on the interaction with the said content, including but not limited to completing tasks, viewing advertisements, interacting with one or more virtual or augmented reality objects during the taxi ride ([0017] “users can be rewarded for interacting with advertising or other content. Such interactions could include, for example, acting or responding to the message such as viewing the message, clicking a link in the message, answering a question, purchasing a product or service, providing feedback, or sending a reply message. In this manner, users can limit access to their information and limit third-party content to those times and places where the user may be interested in viewing the advertising other content. For example, the content could comprise an augmented reality vehicle.” And, [0020] “the user may be rewarded if they chose to view and interact.” And, [0070] “the users could be rewarded for some or all their interactions. Such rewards could include, for example, points, money, discounts, ticket for a drawing, free items or services, and so forth.”)>.
Therefore, it would have been obvious to modify the combination of Couleaud and Melnik to include incentivizing user engagement with augmented reality content by rewarding users with points, vouchers, or redeemable rewards based on the interaction with the said content, including but not limited to completing tasks, viewing advertisements, interacting with one or more virtual or augmented reality objects during the taxi ride as taught by Alghamdi in order “to improve engagement” (Alghamdi [0023]) with the augmented reality content in Couleaud.
Claim 6
As per claim 6, Couleaud further teaches:
interfacing with external data sources, including advertisements platforms and art databases, to procure real time data for generating augmented reality content and advertisements ([0051] “The media overlays may be stored in the database 120 and accessed through the database server.” And [0133] “The SRVMD is configured to provide augmented reality functions, such as overlay information related to a billboard passed by the vehicle.” And, [0106] “The XR environment is an AR or MR environment provided using an AR or MR device.” And, [0129] “acquisition and display of imagery and information pertaining, for example, to roadside information, such as billboards, street name signs, and safety-related objects or information.” And, [0142] “media content is provided (and stored) by an external source (e.g., a third party) separate from the client device.”).
Therefore, it would have been obvious to modify the combination of Couleaud, Melnik, and Alghamdi to include wherein the system incentivizes user engagement with augmented reality content by rewarding users with points, vouchers, or redeemable rewards based on the interaction with the said content, including but not limited to completing tasks, viewing advertisements, interacting with the virtual or augmented reality objects during the taxi ride as taught by Alghamdi in order “to improve engagement” (Alghamdi [0023]) with the augmented reality content in Couleaud.
Claim 16
As per claim 16, Couleaud does not explicitly teach but Alghamdi teaches:
wherein the system offers gamified experiences, challenges, and rewards to encourage user engagement with AR content ([0099] “create a challenge at each pinpoint includes creating AR Games, comment, reply, post, quiz, questions, survey, or the like.” And, [0023] “To improve engagement with messages, recipients can utilize the systems and methods to post media in any location with pinpoints and virtual marks.” And, [0022] “experience the communication messages in different ways like interacting with the message in AR or virtual reality (VR) form, playing interactive games, participating in the experiences and messages and sharing them on social media or other platforms.”).
Therefore, it would have been obvious to modify the combination of Couleaud and Melnik to include wherein the system interfaces with external data sources, such as advertisements platforms and art databases, to procure real time data for generating augmented reality content and advertisements as taught by Alghamdi in order “to improve engagement” (Alghamdi [0023]) with the augmented reality content in Couleaud.
Claim 17
As per claim 17, Couleaud does not explicitly teach but Alghamdi teaches:
wherein the system integrates with social media platforms for sharing AR experiences and content with other users ([0099] “And, [0022] “experience the communication messages in different ways like interacting with the message in AR or virtual reality (VR) form, playing interactive games, participating in the experiences and messages and sharing them on social media or other platforms.” And, [0023] “Account holders would be able to share or sell their AR creations with other account holders.”).
Therefore, it would have been obvious to modify the combination of Couleaud, Melnik, and Alghamdi to include wherein the system integrates with social media platforms for sharing AR experiences and content with other users as taught by Alghamdi in order “to improve engagement” (Alghamdi [0023]) with the augmented reality content in Couleaud.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication 20250139838 (“Couleaud”) in view of US Patent Application Publication Number 20220198701 (“Melnik”) in view of US Patent Application Publication Number 20200258099 (“Alghamdi”) as applied to claim 6 above, and in further view of US Patent Application Publication Number 20240221060 (“Agrawal”).
Claim 7
As per claim 7, Couleaud does not explicitly teach but Agrawal teaches:
wherein the system integrates with external APIs and services to dynamically fetch additional augmented reality content and resources ([0054] “AR device 301 can “talk” to AI system 310 via the AR service API 318, in order to receive augmented reality content provided by an augmented reality service 320 (or other AR applications).”).
Therefore, it would have been obvious to modify the combination of Couleaud, Melnik, and Alghamdi to include wherein the system integrates with external APis and services to dynamically fetch additional augmented reality content and resources as taught by Agrawal in order to “allow for users to experience a customizable composite view of both physical reality and virtual reality by rendering computer-generated graphics over a portion of the user's surrounding physical environment” (Agrawal [0022]).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication 20250139838 (“Couleaud”) in view of US Patent Application Publication Number 20220198701 (“Melnik”) in view of US Patent Application Publication Number 20200258099 (“Alghamdi”) in view of US Patent Application Publication Number 20240221060 (“Agrawal”) as applied to claim 7 above, and in further view of US Patent Application Publication Number 20200173783 (“ANANTHA”)
Claim 8
As per claim 8, Couleaud does not explicitly teach but ANANTHA teaches:
wherein the system integrates with external APIs and services to dynamically fetch additional augmented reality content and resources ([0053] “image recognition algorithm, such as TensorFlow Object Recognition, or Cloud Vision API (both developed by GOOGLE), may be used . . . applications utilizing machine learning or deep learning algorithms may be used to improve and optimize the image recognition features of the system over time.”).
Therefore, it would have been obvious to modify the combination of Couleaud, Melnik, Alghamdi, and Agrawal to include wherein the system integrates with external APis and services to dynamically fetch additional augmented reality content and resources as taught by ANANTHA in order to “improve and optimize the image recognition features of the system over time” (ANANTHA [0053]).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication 20250139838 (“Couleaud”) in view of US Patent Application Publication Number 20220198701 (“Melnik”) in view of US Patent Application Publication Number 20200258099 (“Alghamdi”) in view of US Patent Application Publication Number 20240221060 (“Agrawal”) as applied to claim 7 above, and in further view of US Patent Publication Number 10832456 (“Levy”).
Claim 9
As per claim 9, Couleaud further teaches:
wherein the system displays AR content in various formats, including 2D overlays, 3D models ([0133] “The SRVMD is configured to provide augmented reality functions, such as overlay information related to a billboard passed by the vehicle.” And, [0106] “The XR environment is an AR or MR environment provided using an AR or MR device.” And, [0095] “extended reality (XR) device. XR sessions include at least one of augmented reality (AR), three-dimensional (3D) content, four-dimensional (4D) experiences, virtual reality (VR), mixed reality (MR).” And, [0088] “determine at least one of a 3D structure of the scene.).
Couleaud does not explicitly teach but Levy teaches:
wherein the system displays AR content in various formats, including 2D overlays, 3D models, and spatial audio ([col. 3, lines 35-45] “displaying static or real-time streaming multivariate data onto a 2D computer display, or via virtual reality, mixed reality, augmented reality, 3D glasses or holographic display hardware, or manufactured using hardware such as a 3D printer, the method comprising: generating a 2D shape or 3D object (including 3D extrusion of said 2D shape).” And, [col. 11, lines 1-13] “3D surface may optionally include . . . spatial audio.”).
Therefore, it would have been obvious to modify the combination of Couleaud, Melnik, Alghamdi, and Agrawal to include wherein the system displays AR content in various formats, including 2D overlays, 3D models, and spatial audio as taught by Levy in order to “to improve the computer's operation and functionality by improving the ability to display information” (Levy [col. 1, liesn 55-62]).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication 20250139838 (“Couleaud”) in view of US Patent Application Publication Number 20220198701 (“Melnik”) in view of US Patent Application Publication Number 20200258099 (“Alghamdi”) in view of US Patent Application Publication Number 20240221060 (“Agrawal”) as applied to claim 7 above, and in further view of US Patent Application Publication Number 20210409517 (“Luo”).
Claim 10
As per claim 10, Couleaud does not explicitly teach but Luo teaches:
analyzing user engagement metrics to optimize AR content delivery and user experience ([0121] “analyzing usage metrics and user profile data for the first AR interaction in relation to average usage metrics.” And, [0080] “perform complex image manipulations locally on the client device while maintaining a suitable user experience, computation time, and power consumption.” And, [0131] “collect, store, and analyze interactions with augmented reality content items.” And, [0147] “collects and analyzes usage information about augmented reality content items.”).
Therefore, it would have been obvious to modify the combination of Couleaud, Melnik, Alghamdi, and Agrawal to include analyzing user engagement metrics to optimize AR content delivery and user experience as taught by Luo in order to “bring awareness of the AR content item to the target audience by increasing the weighting of the AR content item” (Luo [0047]).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication 20250139838 (“Couleaud”) in view of US Patent Application Publication Number 20220198701 (“Melnik”) as applied to claim 11 above, and in further view of US Patent Application Publication Number 20120127284 (“Bar-Zeev”).
Claim 12
As per claim 12, Couleaud does not explicitly teach but Bar-Zeev teaches:
wherein the dedicated AR rendering engine supports real-time rendering of AR content based on user location and orientation ([0045] “tracking the orientation and/or location of the user's head, augmented reality images can be rendered from a current perspective of the user, and updated in real-time as the orientation and/or location of the user's head changes.”).
Therefore, it would have been obvious to modify the combination of Couleaud and Melnik to include wherein the rendering engine supports real-time rendering of AR content based on user location and orientation as taught by Bar-Zeev so that “the display and the opacity filter are rendered simultaneously and are calibrated to a user's precise position in space to compensate for angle-offset issues” (Bar-Zeev [0067]).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication 20250139838 (“Couleaud”) in view of US Patent Application Publication Number 20220198701 (“Melnik”) as applied to claim 11 above, and in further view of US Patent Application Publication Number 20170069227 (“Dialameh”).
Claim 13
As per claim 13, Couleaud does not explicitly teach but Dialameh teaches:
further comprising a database of a plurality of recognizable images for AR object recognition and interaction (claim 31 “wherein the recognition database stores recognizable objects.” And, abstract “and a recognition database accessible by the client-server host device or devices. A recognition application residing on the client-server host device or devices is capable of utilizing the recognition database to provide real-time object recognition of visual imagery captured using the portable communication device to the remote user of the portable communication device.”).
Therefore, it would have been obvious to modify the combination of Couleaud and Melink, to include further comprising a database of a plurality of recognizable images for AR object recognition and interaction as taught by Dialameh so that the “device [can] provid[e] object recognition do so reliably, and do so in real-time” (Dialameh [0004]).
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication 20250139838 (“Couleaud”) in view of US Patent Application Publication Number 20220198701 (“Melnik”) as applied to claim 14 above, and in further view of US Patent Application Publication Number 20240221060 (“Agrawal”).
Claim 15
As per claim 15, Couleaud does not explicitly teach but Agrawal teaches:
wherein the system communicates with external services and APIs to retrieve updated AR content and data ([0054] “AR device can “talk” to AI system via the AR service API, in order to receive augmented reality content provided by an augmented reality service (or other AR applications).”).
Therefore, it would have been obvious to modify the combination of Couleaud and Melnik to include wherein the system communicates with external services and APIs to retrieve updated AR content and data as taught by Agrawal in order to “allow for users to experience a customizable composite view of both physical reality and virtual reality by rendering computer-generated graphics over a portion of the user's surrounding physical environment” (Agrawal [0022]).
Claims 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication 20250139838 (“Couleaud”) in view of US Patent Application Publication Number 20220198701 (“Melnik”) in view of US Patent Application Publication Number 20200258099 (“Alghamdi”) as applied to claim 16 above, and in further view of US Patent Application Publication Number 20180276895 (“Hodge”).
Claim 18
As per claim 18, Couleaud does not explicitly teach but Hodge teaches:
wherein the system utilizes augmented reality markers, QR codes, or geolocation data for enhanced object recognition ([0085] “Physical real-world object may include an augmented reality trigger that is recognized by input/output circuitry of augmented reality headset and/or augmented reality subsystem. Examples of augmented reality trigger include but are not limited to barcodes, quick response (QR) codes, or any other predetermined images that can be scanned and recognized by input/output circuitry.” And, [0051] “Augmented reality subsystem 220 performs real-time image recognition on the visual information to identify these objects or the physical location of the user.”).
Therefore, it would have been obvious to modify the combination of Couleaud, Melnik and Alghamdi to include wherein the system utilizes augmented reality markers, QR codes, or geolocation data for enhanced object recognition as taught by Hodge so that “Augmented reality subsystem [can] perform[] real-time image recognition on the visual information to identify these objects or the physical location of the user” (Hodge [0051]).
Claim 19
As per claim 19, Couleaud does not explicitly teach but Hodge teaches:
further incorporating analytics and reporting functionalities to monitor user engagement and performance metrics ([0105] “communication system monitors the user's interactions within augmented reality session through augmented reality device.” And, [0047] “continuously monitor and provide voice data recorded during the augmented reality session to monitoring center.” And, [0063] “monitoring operations to capture audio and/or video of a party and the surrounding physical environment.” And, [0068] “monitoring center may see what users are doing at all times while participating in augmented reality system as well as ascertain the status of the user's physical environment.” And, [0047] “continuously tracks biometric information during an on-going augmented reality session.”)
Therefore, it would have been obvious to modify the combination of Couleaud, Melink, Alghamdi, and Hodge to include further incorporating analytics and reporting functionalities to monitor user engagement and performance metrics as taught by Hodge so that “Augmented reality subsystem [can] perform[] real-time image recognition on the visual information to identify these objects or the physical location of the user” (Hodge [0051]).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication 20250139838 (“Couleaud”) in view of US Patent Application Publication Number 20220198701 (“Melnik”) in view of US Patent Application Publication Number 20200258099 (“Alghamdi”) in view of view of US Patent Application Publication Number 20180276895 (“Hodge”) as applied to claim 19 above, and in further view of US Patent Application Publication Number 20200410766 (“Swaminathan”).
Claim 20
As per claim 20, Couleaud does not explicitly teach but Swaminathan teaches:
wherein the system supports offline AR experiences, caching content for uninterrupted usage in areas with limited internet connectivity ([0265] “to present AR and MR content with high realism. Such techniques, for example, may be used to efficiently update and manage the output of real-time or offline reconstruction and scanning in a mobile device having limited resource and connection to the Internet (continuous or noncontinuous). The techniques make available to a wide variety of mobile AR and MR applications up-to-date.). And, [0277] “3D reconstruction data outside an active area of the device may also be stored in a remote cache, such as in the cloud, which is accessible to the device over a higher latency connection.” And, [0278] “Exchanging 3D reconstruction data between the local cache and remote cache based on a user's predicted movements may ensure that 3D reconstruction data is available with low latency to move into the active memory.”).
Therefore, it would have been obvious to modify the combination of Couleaud, Melink, Alghamdi, and Hodge to include wherein the system supports offline AR experiences, caching content for uninterrupted usage in areas with limited internet connectivity as taught by Swaminathan in order to “efficiently update and manage the output of real-time or offline reconstruction and scanning in a mobile device having limited resource and connection to the Internet (continuous or noncontinuous)” (Swaminathan [0265]).
Response to Arguments
Claim Objections
Applicant’s claim amendments obviate the claim objections raised in the onn-final rejections. However, Examiner notes that additional claim objections have been raised in response to Applicant’s claim amendments (see above).
35 U.S.C. 112(b)
Applicant’s claim amendments obviate the 35 U.S.C. 112(b) rejections raised in then on-final rejections. However, Examiner notes that additional 35 U.S.C. 112(b) rejections have been raised in response to Applicant’s claim amendments (see above).
35 U.S.C. 101 – Software Per Se
Applicant’s claim amendments have obviated the software per se rejection of the claims under 35 U.S.C 101 for claims 11-20. Therefore the rejections have been withdrawn.
35 U.S.C. 101 – Abstract Idea
Applicant's arguments, see pages 10-11, filed 10/14/2025, with respect to the rejection(s) of claims 1-20 under 35 U.S.C. 101 have been fully considered and are persuasive. Therefore the rejections have been withdrawn. Specifically, Examiner agrees that the combination of elements integrates the abstract idea into a practical application.
35 U.S.C. 102/103
Applicant's arguments, see pages 11-16, filed 10/14/2025, with respect to the rejection(s) of claims 1-20 under 35 U.S.C. 102 have been fully considered and are persuasive. Therefore the rejections have been withdrawn. However, upon further consideration, a new ground of rejection is made in view of Couleaud and Melink under 35 U.S.C. 103.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US Patent Application Publication Number 20150032366 (“Man”) discloses personalizing information based on a route that is determined for a particular user
US Patent Application Publication Number 20130278631 (“Border”) discloses users interacting with advertising from AR-based signs and billboards during travel and an advertisement tracking API to monitor advertisement selections
US Patent Application Publication Number 20170330377 (“Akselrod”) disclosing a user in a moving car wearing an augmented reality device and observing a virtual advertisement provided by emitters on a bus moving past the user
US Patent Publication Number 10728236 (“Potes”) discloses advertisement objects placed in public transportation channels (e.g., subways, busses, and taxis) that provide video advertisements to AR device users located within proximity of an object
US Patent Application Publication Number 20090054123 (“Mityagin”) discloses awarding points based on the complexity of the level being played in a game or a number of images reviewed by the user
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALLAN J WOODWORTH, II whose telephone number is (571)272-6904. The examiner can normally be reached Mon-Fri 9:00-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ilana Spar can be reached at (571) 270-7537. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALLAN J WOODWORTH, II/
Primary Examiner, Art Unit 3622