DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen (USPAP 2010/0070,527), in view of Jung et al. (USPAP 2009/0123,021), hereinafter, “Jung”.
Regarding claim 1 Chen teaches, at least one memory storing instructions; and at least one processor configured to access the at least one memory (Please note, paragraph 0013. As indicated storing a plurality of items in a raw data storage, the items comprising images and/or videos, processing the items in a processor.) and execute the instructions to: set, as a candidate region, a region in which an object to be annotated is likely to be present in an annotation target image (Please note, paragraph 0013. As indicated to generate and/or segment annotated information from the items and to extract object, activity and/or metadata information from the items in the first data storage.); generate, as annotation data, data in which the annotation target image, a reference image captured when a region including the candidate region is different from the annotation target image, and the standard image are associated with each other (Please note, paragraph 0010. As indicated providing at least data types of object and activity where the object is a data type representing a thing or a being with visual shape in an image or video frame and the activity is a data type representing an action or an event visually shown in an image or video or video frame such as an explosion, person movement, vehicle movement and so on, providing a function of identify( ) by given a first item to find a list of items that are similar to the first item or to identify the identity of the first item; and the item can be a video or a video frame, or an image or a set of images, or a template extracted from a video or image or images, or an object, or an activity, or an annotated data; and the annotated data is a characteristic or summary representation of its original video or images; and providing a function of compose( ) which creates a complex object from a plurality of existing objects with or without a defined manor, or creates a complex activity from a plurality of existing objects and/or activities; and the manor is an orientation or an order in which objects are located in physically or logically.); and output the generated annotation data. (Please note, figure 1, block 121).
Chen does not expressly teach, extract a standard image that is an image in which an object identical to the object to be annotated is imaged from an annotated image.
Jung teaches, extract a standard image that is an image in which an object identical to the object to be annotated is imaged from an annotated image (Please note, paragraph 0043. As indicated referring to FIG. 3, an example is described in which a word "wedding ceremony", as the user's annotation, is inputted in a particular photo of photos included in a segment #1, and a word "wedding reception", as the user's annotation, is inputted in a particular photo of photos included in a segment #2. The user's annotation analyzer 151 may analyze an identical event and up-down relationship through the user's annotation, "wedding ceremony" and "wedding reception".).
Chen & Jung are combinable because they are from the same field of endeavor.
At the time before the effective filing date, it would have been obvious to a person of ordinary skill in the art to utilize this extract a standard image that is an image in which an object identical to the object to be annotated is imaged from an annotated image of Jung in Chen’s invention.
The suggestion/motivation for doing so would have been as indicated on paragraph 0043, “as a result of the analyzing. Specifically, the user's annotation extraction unit 150 may analyze the identical event and up-down relationship via the user's annotation analyzer 151, and may extract the shared index based on the result of the analyzing via the shared index extraction unit 152.”.
Therefore, it would have been obvious to combine Jung with Chen to obtain the invention as specified in claim 1.
Regarding claim 2 Chen teaches, receive, as annotation information, input of information related to annotation of the object to be annotated on the annotation target image; and in a storage as annotation completion data, the annotation target image and the annotation information in association with each other. (Please note, paragraph 0034. As indicated referring to FIG. 1, when new query language statements get input 101 into the database of objects and activities, a new query language parser 103 will parse the statements into token streams, which are then handed over to a (optional) query Planner 105 for evaluating various possible query paths and choose an optimal one for execution, and trigger internal handling functions in Execution Engine 113 to execute the actual query, the Execution Engine 113 will search through primary storage 115 with indexation (following indexed search path set by Indexation Engine 109) or without indexation (a default maybe-brutal searching path will be followed) to fulfill the query tasks and then generate results 121 and return to user, including pulling out relevant annotated data from secondary data storage 117 and raw data from raw data storage 119 to represent to user as part of query output 121.).
Regarding claim 3 Jung teaches, store, in the storage, an image included in the annotation completion data and information about an imaging position of the image in association with each other (Please note, paragraph 0040. As indicated the database 140 may include an annotation database 141, a photo database 142, and an index database 142, for example. The database 140 may store and maintain a user's annotation, the photo, and the index. Specifically, the annotation database 141 may store and maintain the user's annotation inputted via the user's annotation input unit 111. The photo database 142 may store and maintain the photo encoded by the photo encoding unit 130, and the index database 143 may store and maintain the index generated by the indexing unit 180, for example.); and compare an image obtained by imaging a location identical to the candidate region among images stored in the storage means with an image of the candidate region of the annotation target image to extract the standard image. (Please note, paragraph 0055. As indicated in order to prevent a repeated extraction with respect to a portion of the photos which is not changed compared to a previous photo when analyzing the plurality of photos, the individual situation inference unit 170 may deduce the date and time, the location, the person, the event, the object, a behavior or a pose, and the category with respect to only a changed portion, after the situation change detection unit 160 determines an unchanged portion, for example. As an example, when an individual situation is a date and time, the individual situation inference unit 170 may analyze the date and time information as photo information included in the plurality of photos, and may automatically deduce a user's annotation associated with a source time of the plurality of photos.).
Regarding claim 4 Jung teaches, compare each image of the annotation completion data stored in the storage with the annotation target image to extract the standard image when an image obtained by imaging a location identical to the candidate region is not stored in the storage. (Please note, paragraph 0055. As indicated the individual situation inference unit 170 may analyze the plurality of photos stored in the photo database 142 and may deduce any one or more of a date and time (when), a location (where), a person (who), an event (what occasion), an object (with which thing), a behavior or pose (which action), and a category (which category). Also, in an embodiment, in order to prevent a repeated extraction with respect to a portion of the photos which is not changed compared to a previous photo when analyzing the plurality of photos, the individual situation inference unit 170 may deduce the date and time, the location, the person, the event, the object, a behavior or a pose, and the category with respect to only a changed portion, after the situation change detection unit 160 determines an unchanged portion, for example. As an example, when an individual situation is a date and time, the individual situation inference unit 170 may analyze the date and time information as photo information included in the plurality of photos, and may automatically deduce a user's annotation associated with a source time of the plurality of photos. The source time may refer to a time when the photo is taken, for example. As another example, when the individual situation is a location, the individual situation inference unit 170 may analyze location information, for example, GPS information, as the photo information included in the plurality of photos, and may automatically deduce the user's annotation associated with a source location of the plurality of photos.).
Regarding claim 5 Chen teaches, output, as the standard image, an image in which a determination result at a time of performing annotation is correct and an image in which the determination result is incorrect. (Please note, paragraph 0034. As indicated referring to FIG. 1, when new query language statements get input 101 into the database of objects and activities, a new query language parser 103 will parse the statements into token streams, which are then handed over to a (optional) query Planner 105 for evaluating various possible query paths and choose an optimal one for execution, and trigger internal handling functions in Execution Engine 113 to execute the actual query, the Execution Engine 113 will search through primary storage 115 with indexation (following indexed search path set by Indexation Engine 109) or without indexation (a default maybe-brutal searching path will be followed) to fulfill the query tasks and then generate results 121 and return to user, including pulling out relevant annotated data from secondary data storage 117 and raw data from raw data storage 119 to represent to user as part of query output 121.).
Regarding claim 6 Chen teaches, extract, as a related image, an image of a region related to the candidate region from the reference image (Please note, paragraph 0013. As indicated to generate and/or segment annotated information from the items and to extract object, activity and/or metadata information from the items in the first data storage.); and output as the annotation data, data in which an image of the candidate region, the related image, and the standard image are associated with each other. (Please note, paragraph 0010. As indicated providing at least data types of object and activity where the object is a data type representing a thing or a being with visual shape in an image or video frame and the activity is a data type representing an action or an event visually shown in an image or video or video frame such as an explosion, person movement, vehicle movement and so on, providing a function of identify( ) by given a first item to find a list of items that are similar to the first item or to identify the identity of the first item; and the item can be a video or a video frame, or an image or a set of images, or a template extracted from a video or image or images, or an object, or an activity, or an annotated data; and the annotated data is a characteristic or summary representation of its original video or images; and providing a function of compose( ) which creates a complex object from a plurality of existing objects with or without a defined manor, or creates a complex activity from a plurality of existing objects and/or activities; and the manor is an orientation or an order in which objects are located in physically or logically.).
Regarding claim 7 Jung teaches, set a plurality of the candidate regions by sliding a position of the candidate region on the annotation target image. (Please note, figure 2).
Regarding claim 8 Jung teaches, set the candidate region at a position where the object to be annotated is likely to be present based on map information and a type of the object to be annotated. (Please note, figure 4).
Regarding claims 9, 11-17, analysis similar to those presented for claims 1-8, are applicable.
Regarding claims 10, 18-20, analysis similar to those presented for claims 1-8, are applicable.
Examiner’s Note
The examiner cites particular figures, paragraphs, columns and line numbers in the references as applied to the claims for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claims, other passages and figures may apply as well.
It is respectfully requested that, in preparing responses, the applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMIR ALAVI whose telephone number is (571)272-7386. The examiner can normally be reached on M-F from 8:00-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571)272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format.
For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMIR ALAVI/Primary Examiner, Art Unit 2668 Wednesday, January 14, 2026