Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 5-10 are rejected under 35 U.S.C. 103 as being unpatentable over Babazaki et al. (US 20230011625 A1) in view of Sakai (US 20230186591 A1) and further in view of Nie (A Robust and Efficient Framework for Sports-Field Registration, 2021).
Regarding claim 1, Babazaki et al. discloses an information processing device (e.g., information processing device 4, control unit 17) comprising:
at least one memory (volatile memory, para. 0036, 0038) configured to store instructions; and
at least one processor (one or more processors to execute instructions, para. 0036, 0038) configured to execute the instructions to:
acquire first structural data regarding a first feature point group (structure data storage unit 22 stores structure data regarding the structure of the target structure.
FIG. 2 shows an example of a data structure of structure data. The structure data includes size information and registered feature point information, para. 0043) defined as feature points to be extracted from a target field (structure data storage unit 22
stores structure data regarding the structure of the target structure. FIG. 2 shows
an example of a data structure of structure data. The structure data includes size information and registered feature point information, para. 0042),
acquire feature point information indicating sets of a position on an image and a candidate label thereof indicating a correspondence to a feature point of the first feature point group, the sets being determined based on the image which includes at least a part of the field (feature point output unit 42 outputs, to the structure verification unit 43, a plurality of combinations of the position (also referred to as “structural feature point position Pd”) of each structural feature point in the captured image Im and a label (also referred to as “label Lb”) indicating the class of the each structural feature point, para. 0051);
generate corrected feature point information in which the identified candidate label is corrected to be a label indicating the feature point of the second feature point group; and
Babazaki et al. does not expressly disclose (1) second structural data regarding a second feature point group other than the first feature point group and (2) identify, based on the second structural data, a set of the position on the image and the candidate label corresponding to a feature point of the second feature point group.
Sakai teaches the use of first labels and second labels corresponding to first and second features, paras. 0042, 0053; feature extractor configured based on the feature extractor parameters D1 is trained to output a combination, for each feature point, of the second label and the reliability map or the coordinate value regarding the position of the each feature point in the image when the captured image Im is inputted.
Sakai also teaches that the first label determining unit 43 determines a unique first label for each feature point extracted at step S22 based on the additional information Ia acquired at step S23 (step S24). Thereby, the first label determining unit 43 can suitably generate the second feature point information F2 in which an appropriate first label is assigned to each feature point, para. 0091, the label converting unit 36 may set each second label by receiving an input from the input unit 32 for specifying a set of first labels to be integrated (i.e., first labels associated with the same second label), para. 0081.
Babazaki et al. in view of Sakai are analogous art because they are from the similar problem solving area of assignment of points. At the time of the invention, it would have been obvious to a person of ordinary skill in the art to add the point identification of ambiguous point data of Sakai to the information processing of Babazaki et al. in order to obtain feature point identification of unique points. The motivation for doing so would be to discern feature points.
Babazaki et al. in view of Sakai does not expressly disclose estimate, based on the first structural data, the second structural data, and the corrected feature point information, the field in a coordinate system to be used as a reference by a display device, which is equipped with a camera for capturing the image.
Nie et al. teaches field registration that consists of estimating the image-to-template transformation which is usually parameterized as a homography matrix, section 1. SportsFields which is collected from 192 video-clips from 5 different sports covering large environmental and camera variations.
Babazaki et al. in view of Sakai and further in view of Nie et al. are analogous art because they are from the similar problem-solving area of field registration. At the time of the invention, it would have been obvious to a person of ordinary skill in the art to add the point identification of estimation of Nie et al. to the information processing of Babazaki et al. in view of Sakai in order to obtain feature point identification of unique points. The motivation for doing so would be to estimate field features.
Regarding claim 5, Babazaki et al. discloses the information processing device according to claim 1, wherein a display target field which is the field (tennis court, figure 5A) includes the first feature point group as common feature points also included in a training field displayed in an image for training a feature extractor (training data storage unit 23 stores multiple sets of a training image in which the target structure is photographed and correct answer data indicating the position and label of each structural feature point in the training image, para. 0061), which is used in generating the feature point information, and wherein the display target field is a field whose feature points includes the first feature point group and the second feature point group (feature point information output unit 41 inputs the captured image Im to the identifier, thereby generating structural feature point information “IF”, para. 0050).
Regarding claim 6, Babazaki et al. in view of Sakai discloses the information processing device according to claim 1, wherein a display target field which is the field includes same feature points as a training field displayed in an image for training a feature extractor, which is used in generating the feature point information (training unit 31 performs training of the learning model (identifier) based on the training images, see Babazaki et al., para. 0064), and wherein the second feature point group is a group of candidate positions to be erroneously extracted in generating the feature point information (labels to be attached to feature points existing at symmetrical positions vary, see Sakai, para. 0004).
Regarding claim 7, Babazaki et al. discloses the information processing device according to claim 1, wherein the at least one processor (control unit 17) is configured to execute the instructions to generate, based on an estimation result of the field, coordinate transformation information regarding coordinate transformation between a first coordinate system that is the coordinate system (structure matching unit 43
generates the coordinate transformation information Ic by matching, for each label, the detected position of the structural feature point in the device coordinate system with the position of the structural feature point indicated by the registered position information of the structure data, para. 0054) and a second coordinate system that is a coordinate system adopted in the first structural data and the second structural data (a three-dimensional coordinate system (also referred to as “structural coordinate system”) set with reference to the target structure, para. 0037).
Regarding claim 8, Babazaki et al. discloses the information processing device according to claim 1, wherein the information processing device is the display device for displaying a virtual object superimposed on a scenery (display device 1 superimposes and displays, over or around the target structure, the virtual object that indicates additional information to assist the user in performing sports viewing or theater viewing, para. 0028) and comprises:
a light source (light source 10, para. 0031) configured to emit a display light for displaying the virtual object; and
an optical element (optical element 11, para. 0031) configured to reflect at least a part of the display light to cause an observer to visually recognize the virtual object superimposed on the scenery.
Claim 9, a control method claim, is rejected for the same reason as claim 1.
Claim 10, a non-transitory computer readable storage medium claim, is rejected for the same reason as claim 1.
Allowable Subject Matter
Claims 2-4 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS J LETT whose telephone number is (571)272-7464. The examiner can normally be reached Mon-Fri 9-6 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571) 272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS J LETT/Primary Examiner, Art Unit 2611
Prosecution Insights