Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-6 and 8-16 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nishibe et al (WO 2018131238 A1 – see translation provided in the IDS).
Referring to claim 1:
Nishibe et al disclose an information processing apparatus comprising at least one processor (par. 166: CPU 150) and at least one memory (par. 168-167: ROM 152 / RAM 154) that is in communication with the at least one processor, wherein the at least one memory stores instructions for causing the at least one processor and the at least one memory to acquire a moving image (par. 16: recording device 20 captures a moving image), acquire a feature point group from an image in a range of a space wider than an imaging range at a position where the moving image is captured (par. 64: plurality of feature points identified from the depth map associated with the moving image), and add, to the moving image, position information about the feature point group associated with the captured position (par. 21, 53-54: meta information).
Referring to claim 2:
Nishibe et al disclose the position information about the feature point group is converted so that a position and an orientation of an imaging device by which the moving image is captured is used as a reference (par. 20-21, 64-65: adjusting the display position and the orientation of the moving image so that the depth map and the common plane identified from the depth map associated with the moving image match based on the plurality of feature points identified from the depth map associated with the moving image and the position and the orientation [posture] of the recording device [imaging device]).
Referring to claim 3:
Nishibe et al disclose an imaging device configured to capture the moving image is different from an imaging device configured to capture an image used to acquire the feature point group (par. 16, 64: recording device 20 may also include a depth sensor, a stereo camera, a range finder, or the like to sequentially record a depth map, the plurality of feature points identified from the depth map).
Referring to claim 4:
Nishibe et al disclose acquiring object information from the image used to acquire the feature point group and associate the object information with each feature point of the feature point group (par. 90: object recognition based on a feature point detected from the moving image).
Referring to claim 5:
Nishibe et al disclose acquiring a distance from the imaging device to the object by using the moving image (par. 61: distance between the photographed subject and the recording device 20) and when adding the position information about the feature point group to the metadata, using an order based on a distance to the object associated with each feature point of the feature point group (par. 21, 53-54: the recording position information, synchronously photographed moving image, and depth map from which a plurality of feature points are identified).
Referring to claim 6:
Nishibe et al disclose the object information includes information relating to a likelihood of movement (par. 93).
Referring to claim 8:
Nishibe et al disclose information about the shape of the imaging space includes information about a width, a height, and a depth (par. 64, 68: width and height implied by size).
Referring to claim 9:
Nishibe et al disclose information about the shape of the imaging space includes information about a bearing (par. 64, 66: direction of moving image).
Referring to claim 10:
Nishibe et al disclose acquiring depth information about the moving image (par. 64) and adding the depth information to the moving image (par. 21, 53-54: recording meta information in association with the moving image and the depth map).
Referring to claim 11:
Nishibe et al disclose acquiring the position information about the feature point group by performing simultaneous localization and mapping (SLAM) processing (par. 20, 36).
Referring to claim 12:
Nishibe et al disclose acquiring an image of a real space (par. 16), acquiring a position and an orientation of a camera and position information about a feature point group from the image of the real space (par. 16, 20), and aligning the image of the real space with the moving image based on the position information about the feature point group and the position and the orientation of the camera (par. 20-21, 64-65: adjusting the display position and the orientation of the moving image so that the depth map and the common plane identified from the depth map associated with the moving image match based on the plurality of feature points identified from the depth map associated with the moving image and the position and the orientation [posture] of the recording device [camera], wherein the display condition includes a display area in which a corresponding moving image is displayed in the real space).
Referring to claim 13:
Nishibe et al disclose adding, to the moving image, [in the metadata] a position and an orientation of an imaging device by which the moving image is captured (par. 21).
Referring to claim 14:
Nishibe et al disclose the moving image to which the position information about the feature point group is added is saved as a moving image file (par. 21).
Referring to claim 15:
This claim is the method for performing the corresponding functions of the apparatus as set forth in claim 1 and is therefore rejected for the same reasons as presented above.
Referring to claim 16:
This claim is the product or article of manufacture (i.e., a non-transitory computer-readable storage medium) storing instructions that, when executed by a computer, cause the computer to perform the method as set forth in claim 15 (i.e., the corresponding functions of the apparatus as set forth in claim 1). Nishibe et al (par. 181) and Gao et al (par. 302) discloses such a product for respective steps or functions. Therefore, this claim is rejected for the same reasons as presented above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Nishibe et al as applied to claim 1 above, and further in view of Gao et al (US 20150178930 A1).
Referring to claim 7:
Nishibe et al disclose acquiring information about a shape of an imaging space from the image used to acquire the feature point group (par. 68), but do disclose adding the information about the shape of the imaging space to the moving image, However, Gao et al disclose generating metadata according to a division of the location space into separate regions, including regions of unequal size and/or shape, where a set of observed data points (e.g., object positions) may be used to produce a division scheme that divides the location space such that each separate region contains m observed data points [feature points] (par. 157, 206, 211).
It would have been prima facie obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Nishibe et al in view of Gao et al to add information about the shape of the imaging space to the moving image (in the meta information) in order to further improve tracking of objects in moving images by making video indexing, retrieval, and playback more efficient using metadata tied to object positions and object geometry.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed (i.e., a descriptive title that distinguishes the invention and is not a generic or general description). The new title should take into account any amendments to the claims to best indicate the claimed invention.
The title must be as short and specific as possible (see 37 CFR 1.72(a)), distilling a description of the claimed invention into as few words as possible to capture the essence of the claimed invention. Rather than reciting statutory categories (apparatus, method, product) and some generic descriptor (e.g., information processing), a title that is specific, but characterizes the essence or key aspect(s) of the claimed invention, should be submitted.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12 December 2024 was filed in compliance with the provisions of 37 CFR 1.97 and 1.98. Accordingly, the statement(s) (has / have) been considered by the examiner.
Applicant has provided an explanation of relevance of cited document WO 2018131238 A1 on page 1 of the specification.
Cited Art
The prior art and other references made of record and not relied upon are considered pertinent to applicant's disclosure.
Fukui (US 10949712 B2) disclose an information processing method and an information processing device to provide positional information with higher accuracy in accordance with situations in the real world. The information processing method includes: generating, by a processor, a feature point list in which three-dimensional coordinates of a feature point are associated with a local feature amount of the feature point, the feature point having been detected from observation information collected around a unit area. The information processing device includes: a computation section configured to extract a feature point and a local feature amount related to the feature point from acquired image information; and a communication section configured to acquire a feature point list on a basis of collected observation information.
Hayashi (US 11823394 B2) discloses an apparatus includes an extraction unit configured to extract a plurality of feature information from an image obtained by capturing a real space including a tracking target object, an index estimation unit configured to detect an index arranged on the tracking target object from the image, and estimate a position and orientation of the index, a target object estimation unit configured to estimate a position and orientation of the tracking target object based on the position and orientation of the index and a tracking target model, a classification unit configured to classify the plurality of feature information based on a position and orientation of a camera capturing the real space and the position and orientation of the tracking target object, and a construction unit configured to add feature information determined as belonging to the tracking target object by the classification unit, to the tracking target model.
Huang (US 11928800 B2) disclose an image coordinate system transformation method includes obtaining video images acquired by adjacent cameras, the adjacent cameras including a first camera and a second camera that have an overlapping photography region on a ground plane, recognizing N groups of key points of a target object on the ground plane from the video images acquired by the adjacent cameras, each group of key points including first key point extracted from a video image of the first camera and a second key point extracted from a video image of the second camera, the first key point and the second key point being the same feature point of the same target object appearing in the adjacent cameras at the same moment, and N being an integer greater than or equal to 3, and calculating a transformation relationship between image coordinate systems of the adjacent cameras according to the N groups of key points.
Sagong et al (US 12217454 B2) disclose a sensing data processing device for tracking keypoints within a full observable range of the sensing data processing device based on sensing data collected from a previous frame and a sensing data collected from a current frame through a reference sensor, the full observable range determined based on a combined field of view (FOV) of sensors, manage a new keypoint together with the keypoints in response to the new keypoint being detected, and perform localization of the device based on the managed keypoints. The detection data processing device 100 may exclude keypoints from tracking whose moved position falls outside the overall observation range. The detection data processing device 100 may also remove keypoints from keypoint management that fall outside the observable range (e.g., the overall observation range) among the tracked keypoints.
Nishibe et al (US 20190364224 A1 / US 10868977 B2) disclose an information processing apparatus, an information processing method, and a program that are capable of adaptively displaying a video corresponding to sensed three-dimensional information in a space in which the video is displayed. The information processing apparatus includes: a determination unit that determines a setting related to display of a video when the video corresponding to first three-dimensional information sensed in a first real space is displayed in a second real space, on the basis of a sensing result in the second real space.
Matsumura et al (US 20230108075 A1) disclose a behavior classifier that includes a coordinate estimation unit that receives a plurality of frames in chronological order as an input to estimate a keypoint coordinate group for each of the plurality of frames and to generate a chronological keypoint coordinate group for the plurality of input frames; a matrix generation unit that generates a trajectory matrix and generates a trajectory matrix group in which trajectory matrices are collected for all of keypoints defined for the object; and a behavior classification unit that classifies a behavior of the object based on the trajectory matrix group. Each of the plurality of frames captures an object as a moving image. The keypoint coordinate group is a set of two-dimensional coordinates of a keypoint defined for the object. The chronological keypoint coordinate group is a set of the keypoint coordinate groups arranged in chronological order. The trajectory matrix is a matrix in which two-dimensional coordinates of the keypoint of the chronological keypoint coordinate group are plotted as a trajectory of a curve of the two-dimensional coordinates that is smoothly continuous in chronological order.
Hayashi (JP 2024097690 A) discloses an information processing device that acquires first correspondence information representing the correspondence between each of a plurality of coordinate points in a real space and each of the coordinate points of a first image capturing the real space, and acquires a first position and orientation of the imaging means that captured the first image based on the first correspondence information and feature points included in the first image. The information processing device also sets a search range for a second image captured in real space and having a higher spatial resolution than the first image based on the first correspondence information and the first position and orientation, and acquires second correspondence information representing the correspondence between the first correspondence information and each of the coordinate points of the second image within the search range. Then, the information processing device acquires a second position and orientation of the imaging means that captured the second image based on the second correspondence information and feature points included in the second image. The device prevents the time required for position and orientation estimation processing from becoming long by suppressing the amount of calculation for an image captured by an imaging device with high spatial resolution.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Scott Rogers whose telephone number is 571-272-7467. The examiner can normally be reached 8 am to 7 pm flex.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abderrahim Merouan can be reached on 571-270-5254. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from the Patent Center. Unpublished application information in the Patent Center is available to registered users. To file and manage patent submissions in the Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about the Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Scott A Rogers/
Primary Examiner, Art Unit 2683
27 September 2026
Prosecution Insights