DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
This communication is in response to the Application filed on 11/9/2023.
Claims 1-13 and 15-20 are pending. Claim 14 is withdrawn from consideration.
Election/Restrictions
Applicant’s election without traverse of Group I, claims 1-13 in the reply filed on 1/12/2026 is acknowledged.
PLEASE NOTE: Applicant on page 7 of the Remarks states:
PNG
media_image1.png
140
652
media_image1.png
Greyscale
Examiner would like to point out that the restriction dated 11/12/2025 had claims 15-20 grouped with claims 1-13. Claim 14 was stand alone in a separate species. Therefore claims 1-13 and 15-20 are being examined herewith. Claim 14 is withdrawn from consideration.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 2/27/2025 has been considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-13 and 15-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., abstract idea – mental process) without significantly more. Claim 1 is used as an example. Claims 8 and 15 recite a system and non-transitory computer-readable medium, respectively, having a memory and a processor. The two-part test to identify claims that are directed to a judicial exception (Step 2A) and to then evaluate if additional elements of the claim provide an inventive concept (Step 2B) are:
(1) Are the claims directed to a process, machine, manufacture or composition of matter;
(2A) Prong One: Are the claims directed to a judicially recognized exception, i.e., a law of nature, a natural phenomenon, or an abstract idea;
Prong Two: If the claims are directed to a judicial exception under Prong One, then is the judicial exception integrated into a practical application;
(2B) If the claims are directed to a judicial exception and do not integrate the judicial exception, do the claims provide an inventive concept.
Claim 1. A method comprising: (a) receiving, from a client device, image data; (b) detecting one or more objects depicted by the image data; (c) generating a prompt based on the one or more objects detected within the image data; (d) providing the prompt to a natural language processing model; (e) generating one or more caption options based on the natural language processing model and the prompt; and (f) causing display of a presentation of the one or more caption options at the client device. [emphasis added].
With regard to (1), the instant claims recite an apparatus and a method, therefore the answer is "yes".
With regard to (2A), Prong One: Yes. When viewed under the broadest most reasonable interpretation, the instant claims are directed to a Judicial Exception – an abstract idea belonging to the group of mental process – concepts that are practicably performed in the human mind (including an observation, evaluation, judgement, opinion). The steps of (b), (c), and (e) (above in emphasized claim 1) are generically recited and nothing in these steps precludes the steps from practically being performed by a human equipped with an appropriate apparatus. It can be interpreted as merely looking at the image/data and determining a region/object in the image and writing out a ‘prompt’ related to or describes that region/object. There is nothing in the claim that requires more than an operation that a human, armed with the appropriate apparatus, pen and a paper, cannot perform. The detecting and generating, under its broadest reasonable interpretation, covers performance of the limitation in the mind. The claim encompasses the user looking at data/image once the image is received, region/object such as data with location/time/user information, etc. of a section of the image can be determined. This way, essentially one can present/output information about the section of an image that represents that context. Thus, these limitations are a mental process.
With regard to (2A), Prong Two: No. The instant claims do not apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception of (a) “receiving”, (d) “providing”, and (f) “causing display”, and therefore does not integrate the judicial exception into a practical application.
The use of a system/memory/processor to receive an image (i.e., “data”) at a high level of generality such that said “data” can be used in the operation of the recited judicial exception (the mental step of “receiving”). Providing/supplying “data” does not provide for “integration” of the abstract idea into a practical application, as said data do not change the way in which said system operates. There are no specifics on how the data is received. This can be interpreted as “visualization”. Even if this step is by a “processor” that may be, for example, a camera. A camera/sensor is well known in the field, and receiving data from a camera/sensor is also well known. Even if this step of “providing” the prompt to a natural language processing model, that may be, for example, a human/person. Therefore, once a data/image is provided to a person, a person is able to generate a caption describing what’s in the image or something about said object depicted by the image.
This generic processor limitation is no more than mere instructions to apply the exception using a generic computer component. The ‘causing display of presentation’ is simply outputting/writing out a caption/description about the image or object depicted by the image. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. In conclusion, the claim as a whole does not provide for “integration” of the abstract idea into a practical application.
The claim is directed to the abstract idea.
With regard to (2B), as discussed with respect to Step 2A Prong Two, the additional element in the claim amounts to no more than mere instructions to apply the exception using a generic computer component. The same analysis applies here, i.e., mere instructions to apply an exception using a generic computer component cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B. The pending claims do not show what is more than a routine in the art presented in the claims, i.e., the additional elements are nothing more than routine and well-known steps. There is no improvement to technology here. There are only steps of (b), (c), and (e) with additional elements of (a) “receiving”, (d) “providing”, and (f) “causing display” and it has not been shown that the mental process allows the “technology” to do something that it previously was not able to do.
Therefore, the claims 1, 8, and 15 are ineligible under 35 USC 101.
With regard to dependent claims 2-7, 9-13, and 16-20, similar analysis is applied and therefore does not integrate the judicial exception into a practical application – does not provide significant more than the judicial exception. These claims are similarly rejected for the same reasons discussed in view of steps recited in claim 1 and not repeated herewith.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-13 and 15-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US 2017/0132821 to Valliani et al. (hereafter, “Valliani”).
With regard to claim 1 Valliani discloses a method comprising: receiving, from a client device, image data (paragraphs [0068, 0070] where image data is received); detecting one or more objects depicted by the image data (detecting context by extractor 264, paragraphs [0069, 0076-0078); generating a prompt based on the one or more objects detected within the image data (paragraphs [0079, 0092]); providing the prompt to a natural language processing model (paragraph [0085]); generating one or more caption options based on the natural language processing model and the prompt (paragraph [0085]); and causing display of a presentation of the one or more caption options at the client device (paragraph [0092]).
With regard to claim 2 Valliani discloses wherein the generating the prompt further comprises: accessing contextual data at the client device; and generating the prompt based on the one or more objects detected within the image data and the contextual data (paragraphs [0022, 0040, 0100] where “using user data from other users (i.e., crowdsourcing data) for determining typical user media sharing and caption patterns for events of similar types”).
With regard to claim 3 Valliani discloses wherein the contextual data includes one or more of: location data; temporal data that indicates a time of day; and user profile data (paragraph [0040] where “if many people in a particular location on a particular day are sharing images, then a media-sharing event may be detected and captions automatically generated when a user takes a picture at the location on the particular day.”).
With regard to claim 4 Valliani discloses wherein the generating the prompt further comprises: receiving an input that defines a tone; and generating the prompt based on the one or more objects detected within the image data and the tone defined by the input (paragraphs [0032-0035] where a tone can be interpreted as a demographic information or a particular scenario; also, paragraphs [0056-0059, 0080-0081]).
With regard to claim 5 Valliani discloses wherein the causing display of the presentation of the one or more caption options at the client device further comprises: determining a ranking of the one or more caption options; and causing display of the presentation of the one or more caption options based on the ranking (paragraph [0056] where “first event may be correlated with variables of a second event to identify in-common variables for determining a likely pattern. For example, where a first event comprises, a user posting a digital image of food with a caption from a restaurant on a first Saturday and a second event comprises user posting a digital image with a caption from a different restaurant on the following Saturday, a pattern may be determined that the user posts pictures taken in a restaurant on Saturday.”).
With regard to claim 6 Valliani discloses wherein the generating the prompt based on the one or more objects detected within the image data further comprises: receiving an input that selects an object from among the one or more objects detected within the image data; and generating the prompt based on the object selected by the input (paragraph [0028], “selects a portion of the image that is associated with a recognizable object. The portion of the image may be selected prior to recognition of an object in the image”; paragraph [0071]).
With regard to claim 7 Valliani discloses receiving a request to generate a caption, the request comprising a selection of a graphical icon presented among a set of graphical icons; and generating the prompt to be provided to the natural language processing model responsive to the request that comprises the selection of the graphical icon (paragraph [0092], “presentation component 218 generates user interface features associated with a caption. Such features can include interface elements (such as graphics buttons, sliders, menus, audio prompts, alerts, alarms, vibrations, pop-up windows, notification-bar or status-bar items, in-app notifications, or other similar features for interfacing with a user), queries, and prompts.”).
With regard to claims 8 and 15, claims 8 and 15 are rejected same as claim 1 and the arguments similar to that presented above for claim 1 are equally applicable to claims 8 and 15 A. Valliani discloses a system comprising a memory and a processor as shown in Figures 1-2 and 6, and all of the other limitations similar to claim 1 are not repeated herein, but incorporated by reference.
With regard to claims 9 and 16, claims 9 and 16 are rejected same as claim 2 and the arguments similar to that presented above for claim 2 are equally applicable to claims 9 and 16, and all of the other limitations similar to claim 2 are not repeated herein, but incorporated by reference.
With regard to claims 10 and 17, claims 10 and 17 are rejected same as claim 3 and the arguments similar to that presented above for claim 3 are equally applicable to claims 10 and 17, and all of the other limitations similar to claim 3 are not repeated herein, but incorporated by reference.
With regard to claims 11 and 18, claims 11 and 18 are rejected same as claim 4 and the arguments similar to that presented above for claim 4 are equally applicable to claims 11 and 18, and all of the other limitations similar to claim 4 are not repeated herein, but incorporated by reference.
With regard to claims 12 and 19, claims 12 and 19 are rejected same as claim 5 and the arguments similar to that presented above for claim 5 are equally applicable to claims 12 and 19, and all of the other limitations similar to claim 5 are not repeated herein, but incorporated by reference.
With regard to claims 13 and 20, claims 13 and 20 are rejected same as claim 6 and the arguments similar to that presented above for claim 6 are equally applicable to claims 13 and 20, and all of the other limitations similar to claim 6 are not repeated herein, but incorporated by reference.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 2024/0160797 discloses using computing devices to perform automated operations for automatically generating videos and associated information about a building interior using other visual data about the building interior, as well as presenting the generated videos and associated information in various manners. In some situations, the generation is based at least in part on user input provided via user interactions with a displayed floor plan of the building, such as to select one or more rooms or other areas for which to include visual data in the video, and/or to select one or more building objects and/or other building structural elements and/or other building attributes for which to include visual data in the video. The techniques may further include determining and using information about such building attributes of the building from automated analysis of building information that includes floor plans and acquired building images.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHEFALI D. GORADIA whose telephone number is (571)272-8958. The examiner can normally be reached Monday-Thursday 8AM-6PM, Friday 8AM-12PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at 571-272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
SHEFALI D. GORADIA
Primary Patent Examiner
Art Unit 2676
/SHEFALI D GORADIA/Primary Patent Examiner, Art Unit 2676
Prosecution Insights