DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of elect Invention II in the reply filed on 4/14/2026 is acknowledged.
Claim Objections
Claim 13 is objected to because of the following informalities:
Claim 13 recites “analyze the display” in line 4.
It should be “analyze the display content” because it refers back to line 2.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 13 - 27 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (U.S. Patent Publication 20220179713 A1 ) in view of Wilberding et al. (U.S. Patent Publication 20220159377 A1 Filed: 5/10/2021).
Regarding claim 13, Liu discloses “A generative system comprising:
an input visual stream module to generate a display content; ([0005] Currently, one AI processing process is request-response, for example, a process of image recognition in a smartphone. In the AI processing scenario shown in FIG. 1, a smartphone generates an image (the image may come from a camera of the phone or be downloaded from another channel). The smartphone sends the image to a server by using a network, and the server performs AI processing and sends a result obtained after the AI processing to the smartphone for display. The AI processing process is a typical request-response process. In a server AI processing phase, the smartphone is waiting for the result and cannot perform real-time AI processing on a data stream. [0129] Specifically, in S504, the cloud server inputs the characteristic information of the user and the data stream into a trained network model, obtains a conclusion through inference, and uses the inference conclusion as the AI processing result. A process of AI processing and a form of the result are not specifically limited in this embodiment of this application. [0130] For example, the computing device is a smartphone, the smartphone performs a video playback service, the data stream of the computing device may be a video stream, AI processing may be identifying a person, an object, and a scenario in an image in the video stream, and the AI processing result obtained with reference to the characteristic information of the user may include information such as targeted advertisement pushing and object marking.)
a computational module to receive the display content, the computational module implementing a computer vision task (i) to analyze the display, and (ii) to output a generative display content; ([1030] the data stream of the computing device may be a video stream, AI processing may be identifying a person, an object, and a scenario in an image in the video stream, and the AI processing result obtained with reference to the characteristic information of the user may include information such as targeted advertisement pushing and object marking. [0131] For example, the computing device is an e-book, the e-book performs a text reading service, the data stream of the computing device is a text stream, AI processing may be identifying a text in the text stream of the e-book, and the AI processing result obtained with reference to the characteristic information of the user)
the generative display content is based on the display content and is generated in real time. ([0004] Content of the AI processing results varies in different scenarios. For example, for object recognition software, the AI processing result may be annotation information (such as a physical name or an attribute) of a recognized object. For example, for advertisement push software, the AI processing result may be advertisement push content. [0005] – [0008])
Liu does not disclose “the generative display content is generated in real time”.
Wilberding discloses “the generative display content is generated in real time”. ([0023] generative media content can be dynamically modified in real-time, [0022] Generative media content is content that is dynamically synthesized, created, and/or modified based on an algorithm, whether implemented in software or a physical model. The generative media content can change over time based on the algorithm alone or in conjunction with contextual data (e.g., user sensor data, environmental sensor data, occurrence data). In various examples, such generative media content can include generative audio (e.g., music, ambient soundscapes, etc.), generative visual imagery (e.g., abstract visual designs that dynamically change shape, color, etc.), or any other suitable media content or combination thereof. As explained elsewhere herein, generative audio can be created at least in part via an algorithm and/or non-human system that utilizes a rule-based calculation to produce novel audio content. [0027] [0028] )
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate generate real-time generative display content by Wilberding into device of Liu. The suggestion/motivation would have been to improve efficiency. (Wilberding: [0023])
Regarding claim 14, Liu and Wilberding disclose wherein the computer vision task is selected from a group consisting of depth estimation, semantic segmentation, optical character recognition, object detection, painting, and combinations thereof. (Wilberding [0096] [0103] [0105])
Regarding claim 15, Liu and Wilberding disclose wherein the computer vision task uses a vision model. (Wilberding [0096] [0103] [0105])
Regarding claim 16, Liu and Wilberding disclose where the generative display content is shown on a free-standing display system. (Wilberding [0052] [0071] )
Regarding claim 17, Liu and Wilberding disclose wherein the computer vision task simultaneously operates on metadata of the input video stream and the display content. (Wilberding [0103] [0104] )
Regarding claim 18, Liu and Wilberding disclose wherein the computational module further implements a function to operate on a user input or a user profile. (Wilberding [0023] [0024] [0026])
Regarding claim 19, Liu and Wilberding disclose wherein the computational module comprises a neural network selected from a group consisting of a vision model, a multimodal model, a convolutional neural network, a transformer, a generative adversarial network, a large language model, and combinations thereof. (Liu [0004] [0077])
Regarding claim 20, Liu and Wilberding disclose wherein generative display content is shown as a left-eye image and a right-eye image in a headworn display system. (Wilberding [0023] [0026] [0126])
Regarding claim 21, Liu and Wilberding disclose wherein the generative display content is part of a collaborative visual environment. (Wilberding [0085] [0126])
Regarding claim 22, Liu and Wilberding disclose wherein both the display content and the generative display content are shown simultaneously on an extended display system. (Wilberding [0023] [0024] [0026])
Regarding claim 23, Liu and Wilberding disclose wherein the generative display content is part of a video game and includes an effect chosen form a group consisting of a lighting effect, a particle effect, a depth-based effect, a physics-based effect, an annotation, a head-up display, and combinations thereof. (Wilberding [0023] [0024] [0026])
Regarding claim 24, Liu and Wilberding disclose wherein the display content is a first view of a scene, the computational module further performs a neural radiance field rendering or a Gaussian splatting rendering, and the generative content is a second view of the scene. (Liu [0004] [0077])
Regarding claim 25, Liu and Wilberding disclose wherein the computational module is a first computational module, and further comprising a second computational module implementing a graphics function to blend the generative display content with the display content to show on a display system. (Liu [0004] [0077])
Regarding claim 26, Liu and Wilberding disclose wherein the computational module further comprises a retrieval-augmented generation module coupled to the neural network. (Liu [0004] [0077])
Regarding claim 27, Liu and Wilberding disclose wherein the generative display content is shown on a multilayer display. (Wilberding [0071] [0052])
Claims 30 - 32 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (U.S. Patent Publication 20220179713 A1 ) in view of LaRhette et al. (U.S. Patent Publication 20240281372 A1 Filed: 2/17/2023).
Regarding claim 30, Liu discloses “A generative system comprising:
an input visual stream module to generate a display content; and ([0005] Currently, one AI processing process is request-response, for example, a process of image recognition in a smartphone. In the AI processing scenario shown in FIG. 1, a smartphone generates an image (the image may come from a camera of the phone or be downloaded from another channel). The smartphone sends the image to a server by using a network, and the server performs AI processing and sends a result obtained after the AI processing to the smartphone for display. The AI processing process is a typical request-response process. In a server AI processing phase, the smartphone is waiting for the result and cannot perform real-time AI processing on a data stream. [0129] Specifically, in S504, the cloud server inputs the characteristic information of the user and the data stream into a trained network model, obtains a conclusion through inference, and uses the inference conclusion as the AI processing result. A process of AI processing and a form of the result are not specifically limited in this embodiment of this application. [0130] For example, the computing device is a smartphone, the smartphone performs a video playback service, the data stream of the computing device may be a video stream, AI processing may be identifying a person, an object, and a scenario in an image in the video stream, and the AI processing result obtained with reference to the characteristic information of the user may include information such as targeted advertisement pushing and object marking.)
a computational module to receive the display content, the computational module implementing a computer vision task (i) to analyze the display content, and (ii) to output a generative display content based on the display content, ([1030] the data stream of the computing device may be a video stream, AI processing may be identifying a person, an object, and a scenario in an image in the video stream, and the AI processing result obtained with reference to the characteristic information of the user may include information such as targeted advertisement pushing and object marking. [0131] For example, the computing device is an e-book, the e-book performs a text reading service, the data stream of the computing device is a text stream, AI processing may be identifying a text in the text stream of the e-book, and the AI processing result obtained with reference to the characteristic information of the user)
the generative display content is based on the display content and is generated in real time. ([0004] Content of the AI processing results varies in different scenarios. For example, for object recognition software, the AI processing result may be annotation information (such as a physical name or an attribute) of a recognized object. For example, for advertisement push software, the AI processing result may be advertisement push content. [0005] – [0008])
Liu does not disclose “wherein the generative display content is shown on a free-standing display system.”
LaRhette discloses “wherein the generative display content is shown on a free-standing display system.” ([0244] [0145] [0146] [0130])
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate generate display device by LaRhette into device of Liu. The suggestion/motivation would have been to improve efficiency. (LaRhette: [0244])
Regarding claim 31, Liu and LaRhette disclose wherein the computer vision task is one of motion detection, segmentation, classification, pose estimation, object tracking, 3D reconstruction, depth estimation, optical character recognition, image generation, in-painting, out-painting, and combinations thereof. (LaRhette [0245] [0246])
Regarding claim 32, Liu and LaRhette disclose wherein the computational module comprises a neural network and a retrieval-augmented generation module coupled to the neural network. (LaRhette [0004] [0086] [0090])
Allowable Subject Matter
Claims 1 – 3, 5 – 12 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20240161258 A1 discloses generative image model on [0003]
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHUN-NAN LIN whose telephone number is (571)272-5646. The examiner can normally be reached Monday - Thursday 7:30am - 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benjamin C Lee can be reached at 571-2722963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHUN-NAN LIN/Primary Examiner, Art Unit 2629
Prosecution Insights