Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “deviation detection unit” in claims 1,5,10-15, 19-20, 22 (support in paragraphs [0058, 0062, 0138,0142]; “correction data generation unit” in claims 2-5, 23(support in [0048, 0100, 0102, 0188]); “output unit” in claim 6 ([0041, 0098, 0189]); “imaging device state detection unit” in claim 10 ([0056, 0058, 00121, 0123]); “feature point detection unit” in claim 15 (support in [0051, 0096, 0110]); “feature point matching unit” in claim 15-17 (support in [0051,0058, 0110, 0115]); “feature point map update unit” in claim 18 (support in [0062, 00064, 0066, 0162]); “imaging direction detection unit” in claims 22-25 (support in [0044, 0077, 0082, 0176, 0181]); “metadata generation unit” in claim 22 (support in [0044, 0081, 0177]).
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) s 1-3, 5-7, 12, 14, 20-21 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Omari (US 2019/0182424).
Regarding claim 1, Omari discloses An information processing device comprising a deviation detection unit (fig. 3B(362 processing apparatus), [0057]) that detects a deviation of a first imaging direction based on a captured image, the first imaging direction being an imaging direction of an imaging device that is detected by the imaging device that captures an image to obtain the captured image, the imaging direction being associated with the captured image ([0057, 0074, 0076, 0080], fig. 6 Orientation sensor data is obtained while capturing an image/video. Orientation errors are detected based on orientation setpoint and deviations from the setpoint).
Regarding claim 2, Omari discloses a correction data generation unit that generates, based on the detected deviation, correction data for correcting a reference direction as a reference for the first imaging direction ( [0080, 0083, 0090] fig. 6-7 corrective rotation transformation is performed to obtain to stabilize image with respect to other images; low pass filter smooths out the data to find the trajectory/reference data).
Regarding claim 3, Omari discloses wherein the correction data generation unit outputs the correction data to the imaging device (([0086] mechanical stabilization is performed to adjust orientation of image sensor to the orientation setpoint; [0083] of Omari invoking an EIS module to correct the image for rotation).
Regarding claim 5, Omari discloses wherein the deviation detection unit calculates a reliability of the detected deviation or a reliability of a correction value for the reference direction based on the deviation, and when the reliability is equal to or greater than a predetermined threshold, the correction data generation unit outputs the correction data to the imaging device ([0076] 12 degrees of freedom provide a threshold of reliability in the orientation calculation data; [0063] higher frequency motions are not reliably corrected by the mechanical stabilization system and thus corrected by the EIS module).
Regarding claim 6, Omari discloses comprising an output unit that generates a predetermined output signal including the captured image, and outputs the corrected data in association with the captured image included in the output signal ([0078, 0080] figs. 6-7).
Regarding claim 7, Omari discloses wherein the first imaging direction is detected by integrating an amount of change in orientation of the imaging device with respect to the reference direction ([0078, 0080, 0090]; figs. 6-7 Trajectory is determined from filtered out data. This is the reference direction used to determine orientation errors and determine correction data).
Regarding claim 12, Omari discloses wherein when the captured image satisfies a predetermined condition, the deviation detection unit does not detect the deviation [0095] orientation error may not be determined if it is stored in the buffer and not needed).
Regarding claim 14, Omari discloses wherein the deviation detection unit detects, as the deviation, a difference between the first imaging direction and a second imaging direction that is an imaging direction of the imaging device detected on the basis of the captured image ([0069, 0080, 0083, 0090], orientation error is based on deviations from the trajectory).
Regarding claim 20, Omari discloses wherein the deviation detection unit performs processing of detecting the deviation at a cycle longer than a cycle of capturing the captured image ([0026, 0066, 0083]).
Regarding claim 21, Omari discloses An information processing method comprising detecting, by an information processing device(fig. 3B(362 processing apparatus), [0057]) , a deviation of an imaging direction of an imaging device based on a captured image, the imaging direction being detected by the imaging device that captures an image to obtain the captured image, the imaging direction being associated with the captured image (fig. 6, fig. 7, [0074, 0076, 0080, 0083]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 4, 8-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Omari (US 2019/0182424) in view of Saika (US 2017/0227162).
Regarding claim 4, Omari does not specifically disclose wherein the wherein when the deviation is equal to or greater than a predetermined threshold, the correction data generation unit outputs the correction data to the imaging device.
However, Saika discloses wherein when the deviation is equal to or greater than a predetermined threshold, the correction data generation unit outputs the correction data to the imaging device ([0032, 00119, 0123] when camera error exceeds threshold, stabilization process occurs). It would have been obvious before the filing date of the invention to incorporate the threshold of Saika into the system of Omari in order to avoid unnecessary adjustment for small errors.
Regarding claim 8¸ Saika discloses wherein the first imaging direction is included in metadata output from the imaging device and associated with the captured image ([0064, 0070]).
Regarding claim 9, Saika discloses wherein the metadata further includes lens information that is information on a lens of the imaging device ([0150-0151]).
Regarding claim 10, Saika discloses an imaging device state detection unit that detects a state of the imaging device based on the metadata, wherein when the state of the imaging device satisfies a predetermined condition, the deviation detection unit does not detect the deviation ([0032, 0158] power budget threshold).
Regarding claim 11, Saika discloses wherein when a movement speed of the first imaging direction is equal to or greater than a predetermined threshold, when the imaging device is changing zoom, focus, or iris, when a focal length of the lens is less than a predetermined threshold, or when a depth of field of the lens is less than a predetermined threshold, the deviation detection unit does not detect the deviation ([0115-0122]).
Claim(s) 13, 15-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Omari (US 2019/0182424) in view of Rinner (US 2012/0050524).
Regarding claim 13, Omari does not specifically disclose wherein when a distribution of feature points in the captured image is biased or when a ratio of a region of a moving object in the captured image is equal to or greater than a predetermined threshold, the deviation detection unit does not detect the deviation.
However, Rinner discloses wherein when a distribution of feature points in the captured image is biased or when a ratio of a region of a moving object in the captured image is equal to or greater than a predetermined threshold, the deviation detection unit does not detect the deviation ([0063,0068,0069,0115], figs. 5, 7 Only feature points that are within a predefined maximum distance of relevance to a reference plane are considered). It would have been obvious before the filing date of the invention to incorporate the feature point matching of Rinner into the system of Omari in order to provide another method of detecting errors.
Regarding claim 15, Rinner discloses a feature point detection. unit that, detects feature points in the captured image; and a feature point matching unit that makes a match between feature points in a feature point map indicating a distribution of the feature points around the imaging device and the feature points in the captured image, wherein the deviation detection unit detects the deviation based on a result of making a match between the feature points in the feature point, map and the feature points in the captured image ([0063, 0068, 0069, 0115]; figs. 5, 7).
Regarding claim 16, Rinner discloses wherein the feature point matching unit makes a match between the feature points in the feature point map corresponding to the first imaging direction and the feature points in the captured image( ([0088, 0096, 0112])).
Regarding claim 17, Rinner discloses wherein the feature point matching unit makes a match between the feature points in the feature point map and feature points in the captured image excluding feature points in a region of a moving body([0112, 0131]).
Regarding claim 18, Rinner discloses a feature point map update unit that updates the feature point map based on the feature points in the captured image ([0048, 0054]; figs. 5, 7).
Regarding claim 19, Rinner discloses wherein the deviation detection unit detects the second imaging direction based on a result of making a match between the feature points in the feature point map and the feature points in the captured image([0063, 0068, 0069, 0115]; figs. 5, 7).
Claim(s) 22-23, 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Omari (US 2019/0182424) in view of Huang (US 2015/0341536).
Regarding claim 22, Omari discloses An information processing system comprising: an imaging device that captures an image to obtain a captured image; and an information processing device that controls the imaging device, wherein the imaging device includes an imaging direction detection unit that detects an imaging direction of the imaging device ([0074, 0076, 0080], fig. 6, fig. 7).
Omari does not specifically disclose a metadata generation unit that generates metadata including the imaging direction detected by the imaging direction detection unit, and outputs the metadata in association with the captured image to the information processing device, and the information processing device includes a deviation detection unit that detects, based on the captured image, a deviation of the imaging direction included in the metadata.
However, Huang discloses a metadata generation unit that generates metadata including the imaging direction detected by the imaging direction detection unit, and outputs the metadata in association with the captured image to the information processing device, and the information processing device includes a deviation detection unit that detects, based on the captured image, a deviation of the imaging direction included in the metadata ([0058, 0060, 0061, 0066, 0076,0079, 0081, 0082, 0090), fig. 3] Orientation data is stored as metadata during image/video capture. Deviations from target orientation are detected). It would have been obvious before the filing date of the invention to incorporate the metadata information of Huang into the system of Omari in order to use sensor metadata to detect deviations/errors.
Regarding claim 23, Omari in view of Huang discloses wherein the information processing device further includes a correction data generation unit that generates, based on the detected deviation, correction data for correcting a reference direction as a reference of the imaging direction, and outputs the correction data to the imaging device, and the imaging direction detection unit corrects the reference direction based on the correction data ([0074, 0076, 0080, 0083, 0087], figs. 6-7 of Omari).
Regarding claim 25, Omari in view of Huang discloses wherein the imaging direction detection unit continuously corrects the reference direction in a plurality of times ([0074, 0076, 0080, 0083, 0087], figs. 6-7 of Omari).
Claim(s) 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Omari (US 2019/0182424) in view of Huang (US 2015/0341536) in view of Saika (US 2017/0227162).
Regarding claim 24, Omari in view of Huang does not specifically disclose wherein when a movement speed of the imaging direction is equal to or greater than a predetermined threshold, the imaging direction detection unit corrects the reference direction.
However, Saika discloses wherein when a movement speed of the imaging direction is equal to or greater than a predetermined threshold, the imaging direction detection unit corrects the reference direction ([0119,0123,0144]). It would have been obvious before the filing date of the invention to incorporate the speed assessment of Saika into the system of Omari in view of Huang in order to make correction determinations based on the speed.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL HYUN HONG whose telephone number is (571)270-1553. The examiner can normally be reached M-F 9:00-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nasser Goodarzi can be reached at (571)272-4195. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL H HONG/ Primary Examiner, Art Unit 2426