Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The applicant has amended claims 1 and 12-16. The application has pending claims 1-20.
Response to Arguments
Applicant’s arguments with respect to claims 1 and 12-16 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The new reference of SU, in combination with Ayyar and Venkata, disclose meet each of the new limitations of the claims as disclosed in the rejection below. Thus, this action is made FINAL.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 12-16 are rejected under 35 U.S.C. 103 as being unpatentable over Balan Rama Ayyar US-20190073520-A1, hereinafter Ayyar, in combination with Ja-Hwung Su Efficient Relevance Feedback for Content-Based Image Retrieval by Mining User Navigation Patterns, hereinafter SU, and Venkata N. Padmanabhan US-20030176965-A1, hereinafter Venkata.
As per claim 1, Ayyar discloses an information processing apparatus comprising:a storage device that stores instructions; andat least one processor that executes the instructions to (see Ayyar ¶27, wherein a processor performing instructions is disclosed):acquire a captured image (see Ayyar ¶31-32, wherein a digital file, such as a video or image, is received);detect a plurality of targets included in the captured image, and extract a plurality of features for each of the detected plurality of targets (see Ayyar ¶37, wherein unidentified individuals are detected in the frame, and features of that person, or object, is extracted as further disclosed in ¶40);acquire an impurity for each feature of the extracted plurality of features, the impurity indicating a degree to which a predetermined target is unrecognizable in a case where a user is asked a question for presuming the predetermined target from among the plurality of targets based on each feature (see Ayyar ¶44, wherein the user is queried and unidentified targets are matched against the plurality of known features of individuals or objects. The query processor also extracts features of a target. See also Ayyar ¶42-43 and further Ayyar ¶46, wherein a recognition module uses a confidence threshold, i.e., impurity value, for the connection amongst the targets is disclosed).
However, Ayyar fails to explicitly disclose where SU teaches:the impurity indicating a degree to which a predetermined target is inseparable from among the plurality of targets based on an answer from a user for a question (see SU page 6/23 and FIG. 5, wherein the user feedback is given to the positive example images. The system then asks if the user is satisfied with their result. If no, the user feedback is sent back into the image search phase and query re-weighting is performed).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar’s apparatus by using SU’s teaching by modifying impurity to a degree where a target is inseparable from a plurality of targets to the impurity extraction using an answer from the user in order to improve and specify object recognition by comparing the targets amongst themselves.
However, Ayyar, in combination with SU, fails to explicitly disclose where Venkata teaches:
generate the question related to a feature (see Venkata ¶72, wherein the question-and-answer feedback cycle asks the user questions about the landmarks, the landmarks comprising features disclosed in ¶45) with smaller impurity compared to impurity in other features (see Venkata ¶71-72, in the area with all the candidate landmarks, the question asked to the user in order to reduce the listed landmarks which are underneath the confidence threshold. The landmarks that remain in the list after the question have a higher confidence, i.e., smaller impurity, to those removed off the list for being under the threshold), to reduce a number of questions for minimizing the impurity (see Venkata ¶67 and ¶72, wherein query optimization: sufficient confidence is achieved by reducing the candidate landmarks list by such a question-and-answer feedback cycle. The query optimization only considers the buildings within the search area, so it does not need to ask questions irrelevant to the visibility region) based on the plurality of features extracted for the plurality of targets and the respective impurity for each feature of the extracted plurality of features (see Venkata ¶72, wherein the question-and-answer feedback cycle asks specific questions when determining the user’s location. The cycle continues until each of the candidate landmarks described by the user is reduced to the result(s) with sufficient confidence, i.e., smallest impurity).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar’s, in combination with SU, apparatus by using Venkata’s teaching by including an optimized question generation process to Ayyar’s apparatus in order to reduce the questions asked to the user for a faster conclusion.
As per claim 12, the rationale provided in claim 1 is incorporated herein. In addition, Ayyar discloses a mobile object (see Ayyar ¶97, wherein a smartphone is disclosed).
As per claim 13, the rationale provided in claim 1 is incorporated herein. In addition, Venkata discloses a control method of an information processing apparatus (see Venkata ¶22-25, wherein a computer, i.e., information processing apparatus, controlled by a user is disclosed).
As per claim 14, the rationale provided in claim 1 is incorporated herein. In addition, Ayyar teaches a control method of a mobile object (see Ayyar ¶96-97, wherein the processor or controller for a machine, such as the smartphone, is disclosed).
As per claims 15-16, the rationale provided in claim 1 is incorporated herein. In addition, Ayyar discloses a non-transitory storage medium (see Ayyar ¶96, wherein a non-transitory storage medium is disclosed).
Claims 2, 4, 6, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Ayyar, in combination with SU and Venkata, in further view of Robin DUA US-20170185670-A1, hereinafter DUA.
As per claim 2, Ayyar, in combination with SU and Venkata, fails to explicitly disclose where DUA teaches:
the information processing apparatus according to claim 1, wherein the at least one processor executes instructions in the storage device to:extract the features using an image recognition model (see DUA ¶56, wherein the image processing module, i.e., image recognition model, performs image recognition using features such as cars, dress, animals, characteristics, etc.), and generate the question that minimizes the impurity in a shortest time based on a reliability and a weight of the features extracted using the image recognition model in addition to the features and the impurity (see DUA ¶75-80, wherein the indexing module, which is coupled with the image processing module as shown in FIG. 2, uses a confidence score and label weights in order to improve the search query and reduces computational time).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar’s, in combination with SU and Venkata, apparatus by using DUA’s teaching by modifying the question generation to include a shortest time reliability and feature weight in order to more rapidly select the features with higher weights to minimize impurity.
As per claim 4, Ayyar, in combination with SU, Venkata, and DUA, discloses the information processing apparatus according to claim 2, wherein the weight is set, for each feature, heuristically or based on machine learning (see DUA ¶76, wherein a propagation algorithm, i.e., heuristics, is used for defining edge weights).
As per claim 6, Ayyar, in combination with SU and Venkata, fails to explicitly disclose where DUA teaches:
the information processing apparatus according to claim 1, wherein the at least one processor executes instructions in the storage device to:transmit a generated question to a communication device possessed by the user (see DUA ¶109, wherein the user interface module generates an interface that displays the question. See also FIG. 3A, wherein a phone, i.e., communication device, is shown);receive an answer to the question from the communication device (see DUA ¶106 and FIG. 5, wherein "yes" and "no" buttons are displayed on the screen for the user to interact with); andpresume the predetermined target from among the plurality of targets according to the received answer (see DUA ¶109, wherein the image assistant can access images of a target based on an answer to a question).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar’s, in combination with SU and Venkata, apparatus by using DUA’s teaching by include a transmitted question via a communication device to the question-and-answer feedback in order to further improve on the communication between the user and the device.
As per claim 18, Ayyar, in combination with SU and Venkata, fails to explicitly disclose where DUA teaches:
the information processing apparatus according to claim 1, wherein the question is generated with a format accepting a binary input from the user (see DUA ¶106 and FIG. 5, wherein "yes" and "no" buttons, i.e., binary input, are displayed on the screen for the user to interact with).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar’s, in combination with SU and Venkata, apparatus by using DUA’s teaching by including a binary input to the question generation in order to acquire a simple and precise answer regarding the question.
As per claim 19, Ayyar, in combination with SU, Venkata, and DUA, discloses the information processing apparatus according to claim 2, wherein the at least one processor executes instructions in the storage device to generate the question related to the feature which only one user has (see DUA ¶108, wherein the question is generated to the user to confirm the person, such that there is only one confirmed person. See prior ¶27, wherein the features are extracted for the user information).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Ayyar, in combination with SU, Venkata, and DUA, in further view of Richard KWANT US-20190102656-A1, hereinafter KWANT.
As per claim 3, Ayyar, in combination with SU, Venkata, and DUA, fails to explicitly disclose where KWANT teaches:the information processing apparatus according to claim 2, wherein the reliability indicates a reliability of a feature value indicating a value of a feature extracted by the image recognition model for each of the plurality of targets (see KWANT ¶52-53, wherein a QA criterion, i.e., reliability of a feature value, for each of the data items obtained by the feature detector).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar’s, in combination with SU, Venkata, and DUA, apparatus by using KWANT’s teaching by modifying the image recognition model to Ayyar’s information processing apparatus in order to more accurately query and more precisely discern the feature values of the captured targets.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Ayyar, in combination with SU, Venkata, and DUA, in further view of Cristian Alb US-20220156523-A1, hereinafter Alb.
As per claim 5, Ayyar, in combination with SU, Venkata, and DUA, fails to explicitly disclose where Alb teaches:the information processing apparatus according to claim 2, wherein the impurity is acquired according to at least one or more of a number of targets excluding the predetermined target included in a set of the plurality of targets, and a penalty that is based on the weight and/or the reliability of the feature (see Alb ¶12 and ¶16, wherein a target column is set and all other target sets are compared against the target using entropy or other impurity measures).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar, in combination with SU, Venkata, and DUA, by using Alb’s teaching by including a number of targets to the image processing apparatus in order to more accurately determine the impurity by comparing the other targets to the predetermined target.
Claims 7 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Ayyar, in combination with SU, Venkata, and DUA, in further view of Shoashan Liu US-10794710-B1, hereinafter Liu.
As per claim 7, Ayyar, in combination with SU, Venkata, and DUA, fails to explicitly disclose where Liu teaches:the information processing apparatus according to claim 1, wherein the image acquisition unit acquires position information from a communication device possessed by the user, and acquires a captured image of surroundings of the position information from outside (see Liu col. 4 lines 7-54, wherein the cameras capture the surrounding area as well as absolute or relative position information. See further col. 16 lines 20-27, wherein user input for transition signals using a signal processor, i.e., communication device, is disclosed).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar’s, in combination with SU, Venkata, and DUA, apparatus by using Liu’s teaching by including position information from a captured device by the user to the image processing apparatus in order to further obtain positional information of the area in which the targets will be identified and extracted.
As per claim 9, Ayyar, in combination with SU, Venkata, DUA, and Liu, discloses the information processing apparatus according to claim 7, wherein the at least one processor executes instructions in the storage device to acquire a captured image captured by a camera installed around the position information from the camera (see Liu col. 4 lines 7-54, wherein the cameras capture the surrounding area as well as relative position information).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Ayyar, in combination with SU, Venkata, DUA, and Liu, in further view of Ali Haghighat Kashani US-20220005358-A1, hereinafter Kashani.
As per claim 8, Ayyar, in combination with SU, Venkata, DUA, and Liu, fails to explicitly disclose where Kashani teaches:the information processing apparatus according to claim 7, wherein the image acquisition unit acquires an image captured by a vehicle that the user requests to join from the vehicle (see Kashani ¶29-30 and more specifically ¶76, wherein the user requests to join the vehicle. See also ¶73-74, wherein the imagine sensors of the vehicle are used to obtain data of its surroundings. See further FIGS. 6-7).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar’s, in combination with SU, Venkata, DUA, and Liu, apparatus by using Kashani’s teaching by including an image captured by the vehicle to the image acquisition unit in order to obtain better informative data of the surrounding area.
Claims 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Ayyar, in combination with SU, Venkata, and DUA, in further view of CHONG HIN US-20210319765-A1, hereinafter HIN.
As per claim 10, Ayyar, in combination with SU, Venkata, and DUA, fails to explicitly disclose where HIN teaches:The information processing apparatus according to claim 1, wherein in a case where the target is a person, the feature is at least one piece of information selected from a group of pieces of information indicating a nearby object, clothes color, clothes type, bag color, bag type, whether the person is looking at a communication device, and/or whether the person is wearing a mask (see HIN ¶31, wherein the target is a person and includes metadata about their clothes, any article of clothing, such as a mask, any inanimate object features such as their bag type and color, and the information of the surrounding area).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar’s, in combination with SU, Venkata, and DUA, apparatus by using HIN’s teaching by including a case where the target is a person to the information processing apparatus in order to obtain more detailed information about the target if it is identified to be a person.
As per claim 11, Ayyar, in combination with SU, Venkata, and DUA, fails to explicitly disclose where HIN teaches:The information processing apparatus according to claim 1, wherein the feature is at least one piece of information selected from a group of color of the target, type of the target, a character shown on the target, and/or a pattern shown on the target (see HIN ¶31, wherein the type of the target, object or person, the target’s color, tattoos or birthmarks, i.e., character shown on the target, and pattern of jewelry on the target are disclosed).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar’s, in combination with SU, Venkata, and DUA, apparatus by using HIN’s teaching by including a case where the target is a person to the information processing apparatus in order to obtain more detailed information about the target.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Ayyar, in combination with SU, Venkata, and DUA, in further view of Bin Cheng US-20220036098-A1, hereinafter Cheng.
As per claim 17, Ayyar, in combination with SU, Venkata, and DUA, fails to explicitly disclose where Cheng teaches:The information processing apparatus according to claim 1, wherein the at least one processor executes instructions in the storage device to:acquire position information of the predetermined target from the user (see Cheng ¶29 and more specifically ¶42, wherein the position information of the object relative to the ego vehicle, i.e., the user, is acquired), andgenerate a question according to the feature amount which the predetermined target is likely to have (see Cheng ¶41, wherein the image database is queried using features of the object).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar’s, in combination with SU, Venkata, and DUA, by using Cheng’s teaching by including question generation using position information to the information processing apparatus in order to create more personalized questions using the positional information relative to the user.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Ayyar, in combination with SU, Venkata, and DUA, in further view of YATAKE HIDENORI JP-2005037615-A, hereinafter HIDENORI.
Ayyar, in combination with SU, Venkata, and DUA, fails to explicitly disclose where HIDENORI teaches: The information processing apparatus according to claim 1, wherein the at least one processor executes instructions in the storage device to transmit the question again or be terminated with error, in a case where the question is not answered after a predetermined time period has elapsed from the transmission of the question (see HIDENORI ¶90-91, wherein an error message is displayed if the user does not recognize the user’s response to the question, as disclosed in ¶78, or after a predetermined time has elapsed).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Ayyar’s, in combination with SU, Venkata, and DUA, apparatus by using HIDENORI’s teaching by including a time out error to the information processing device in order to guarantee an answer from the user and to prevent continuing the query process without a valid response from the user.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Bradley Obas Felix whose telephone number is (703)756-1314. The examiner can normally be reached M-F 8-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 5712728243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRADLEY O FELIX/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights