DETAILED ACTION
Response to Arguments
Applicant’s arguments, filed 8/26/2026, with respect to how the claim features from independent claim 1 differ from the prior art cited in the last office have been fully considered. These arguments are found to be persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in this office action.
Allowable Subject Matter
Claims 2, 11, 13-14, 19, and 20-21, 23 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1, 17, 22, and 30 is rejected under 35 U.S.C. 103 as being unpatentable over Choe et al. (US 20220301184 A1).
As per claim 1, Choe teaches the claimed:
1. A method comprising:
obtaining a first image and a second image (Please see the top of figure 8 where it shows frames N and N+1 as first and second images);
determining that the first image and second image satisfy a first similarity criterion (In figure 12, where the first similarity criterion is determined at step 1204 “Perform Alignment and Post-Processing”. Also, please see [0076] where it explains that this step uses similarity criterion, e.g. by considering alignment using optical flows between frames (e.g. first and second images) and by considering how to reduce erosion artifacts between frames);
responsive to determining that the first image and second image satisfy the first
similarity criterion, determining that the first image and second image satisfy a second
similarity criterion (In figure 12, where the second similarity criterion is determined at step 1206 “Use Solver to Identify Non-Linear Curve”. This is in response to first performing an alignment (first similarity criterion) in step 1204. It is noted that the solver to identify the non-linear curve needs an alignment process to be performed ahead of time for more accurate results (e.g. please see in [0075] “The alignment operation 800 is generally used to improve slow-motion or other interpolated image quality, such as in areas of frames where non-linear motion (like a jumping dog) occurs between neighboring frames.”) Thus, the second similarity criterion is determined in response to the first similarity criterion already being established.
In this instance, the identification of the non-linear curve is a second similarity criterion because it is considering how similar frames jump between each other in space, e.g. please see paragraph [0072] “The equation solver 720 generally operates to identify linear, quadratic, cubic, or higher-order polynomial curves that define how the positions of pixels change between different video frames 740.”), wherein determining that the first image and second image satisfy the second similarity criterion has a higher computational cost than determining that the first image and second image satisfy the first similarity criterion (The identification of the non-linear curve has a higher computational cost because it requires the use of a solver. It is obvious to one of ordinary skill in the art that mathematical problems that require the use of a separate solver to solve non-linear curve fittings will typically have higher computational costs compared to other steps that do not require a specialized separate solver); and
responsive to determining that the first image and second image satisfy the second
similarity criterion, generating a video sequence based on the first image and second image,
wherein the video sequence comprises a plurality of interpolated images (End of [0006] “… The method also includes generating interpolated video frames between the first and second video frames by applying non-linear interpolation based on the non-linear curve. In addition, the method includes outputting the interpolated video frames for presentation”. Also, please see figure 12 in step 1208. In this figure, step 1208 first needs a non-linear curve to be identified (responsive the second similarity criterion)) corresponding to respective different points in time between a first time associated with the first image and a second time associated with the second image ([0088] “FIG. 12 illustrates an example method 1200 for accurate optical flow interpolation optimizing bi-directional consistency and temporal smoothness according to this disclosure.” Also, please see in figure 9 where the diamond shaped symbols show that the interpolated frames correspond to respective different points in time (e.g. frames 0.25, 0.5, 0.75) between the first time at the first image (frame 0) and second time at the second image (frame 1)).
As per claim 30, this claim is similar in scope to limitations recited in claim 1, and thus is rejected under the same rationale. The system of Choe teaches of a non-transitory machine readable medium in paragraph [0011].
As per claim 17, Choe teaches the claimed:
17. The method of claim 1, wherein at least one of determining that the first image and second image satisfy the first similarity criterion or determining that the first image and second image satisfy the second similarity criterion comprises:
determining a first optical flow pattern from the first image to the second image; and
determining a second optical flow pattern from the second image to the first image.
(Choe [0060]: “Bi-directional interpolation combines forward optical flow interpolation as shown in FIG. 2 and backward optical flow interpolation as shown in FIG. 3. The forward optical flow interpolation process estimates pixel locations in a subsequent frame based on known pixel locations in an earlier frame, and the backward optical flow interpolation process estimates pixel locations in an earlier frame based on known pixel locations in a subsequent frame.” Choe teaches the forward and backward optical flow interpolation, described in paragraphs [0056] and [0059] respectively, that estimates pixel location from an earlier frame, the first image, to a subsequent frame, the second image, and vice versa.)
As per claim 22, Choe teaches the claimed:
22. The method of claim 1, wherein at least one of determining that the first image and second image satisfy the first similarity criterion or determining that the first image and second image satisfy the second similarity criterion comprises:
determining a mutual information between the first image and the second image (In figure 12 where mutual information is considered between the first and second frames (images) for both the alignment process and for determining a non-linear curve, e.g. please see [0072] and [0076]).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Choe in view of Li (US 20210056307 A1).
As per claim 3, Choe alone does not explicitly teach the claimed limitations.
However, Choe in combination with Li teaches the claimed:
3. The method of claim 1, wherein determining that the first image and second image satisfy a first similarity criterion comprises at least one of:
determining that the first time associated with the first image and the second time associated with the second image differ by less than a threshold time difference,
determining that a photo quality score of the first image and a photo quality score of the second image differ by less than a threshold amount, or
determining that a content type of the first image and a content type of the second image differ by less than a threshold amount.
(Li [0046]: “one or more of the constituent similarity scores may be calculated first for a given pair of objects depicted across two images. If those one or more constituent similarity scores satisfy some minimum threshold (e.g., 0.9), then the analysis may end and the two objects may be determined to be a match.”
Li [0047]: “A similarity score sim.sub.2d may be determined in various ways. … Various feature points within the bounding shapes, such as edges, volumes, shapes, patterns, gradients, sizes, textures, colors, visible feature counts, etc., may then be compared to determine a similarity between the contents of the two respective bounding shapes.” Li teaches comparing the similarity between the contents of two images and determining a similarity score based on whether a threshold is met or not.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the similarity score that determines a threshold amount as taught by Li with the system of Choe in order to improve accuracy of aligning objects in an image.
Claims 4 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Choe in view of Wang (US 20130148892 A1).
As per claim 4, Choe alone does not explicitly teach the claimed limitations.
However, Choe in combination with Wang teaches the claimed:
4. The method of claim 1, wherein at least one of determining that the first image and second image satisfy the first similarity criterion or determining that the first image and second image satisfy the second similarity criterion comprises:
determining that a first multidimensional embedding that represents contents of the first image differs from a second multidimensional embedding that represents contents of the second image by less than a threshold amount.
(Wang [0028]: “the control unit 4 detects plural feature points P.sup.1 (hereinafter referred to as first feature points P.sup.1) from the first document image 14a and feature counts C.sup.1 representing the image features at the respective feature points P.sup.1”
Wang [0034]: “Specifically, the control unit 4 calculates a Euclidean distance between the feature count C.sup.2.sub.l (the multi-dimensional vector) at the second feature point P.sup.2.sub.l and the feature count C.sup.1.sub.i (the multi-dimension vector) with respect to each of the second feature points P.sup.2.sub.l and specifies the second feature points P.sup.2.sub.l of which the Euclid distance is a threshold value or less as the second feature point candidates P.sup.2.sub.j.” Wang teaches the control unit that detects first and second feature points and compares the distance between the first and second multi-dimensional vectors based on a threshold value.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the control unit as taught by Wang with the system of Choe in order to calculate and compare the distance between multi-dimensional vectors.
As per claim 6, Choe and Wang teach the claimed:
6. The method of claim 4, further comprising:
determining a first location of a first salient region within the first image; and
(Wang [0030]: “the control unit 4 performs a known character region detecting process … in the first document image … the control unit 4 specifies a region occupied by a circumscribed rectangle” Wang teaches the character region detecting process in the first image, which corresponds to the determination of a region within the first image.)
determining the first multidimensional embedding based on the first image in a manner that preferentially weights contents of the first salient region within the first image.
(Wang [0036]: “The first feature point acquiring unit 16 acquires plural first feature points P.sup.1.sub.i included in any of the character regions in the first document image 14a and feature counts C.sup.l.sub.i at these first feature points P.sup.l.sub.i”. Wang teaches the feature count, also the multi-dimensional vectors, based on the feature points in the first image and the character regions.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the control unit as taught by Wang with the system of Choe in order to calculate and compare the distance between multi-dimensional vectors.
Claims 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Choe and in view of Komatsu (US 20150227815 A1).
As per claim 8, Choe alone does not explicitly teach the claimed limitations.
However, Choe in combination with Komatsu teaches the claimed:
8. The method of claim1, wherein at least one of determining that the first image and second image satisfy the first similarity criterion or determining that the first image and second image satisfy the second similarity criterion comprises:
determining that a pixelwise distance between the first image and the second image is less than a threshold distance.
(Komatsu [Abstract]: “A distance measurement apparatus that calculates distance information based on a first and second images, including: a first calculation unit that calculates, for each pixel for distance calculation … a determination unit that determines whether the non-similarity is higher than a threshold”. Komatsu teaches the determination of a pixel distance between the first and second image to see if the non-similarity is higher than a threshold. Thus, it would be obvious to say if what is higher than a threshold is known, then what is less than that threshold can also be determined.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the calculation unit as taught by Komatsu with the system of Choe in order to be able to calculate and determine the pixel distance between multiple images.
As per claim 9, Choe and Komatsu teach the claimed:
9. The method of claim 8, wherein determining that a pixelwise distance between the first image and the second image is less than a threshold distance comprises:
determining a first location of a first salient region within the first image; and
determining the pixelwise distance between the first image and the second image in a manner that preferentially weights contents of the first salient region within the first image.
(Komatsu [0013]: “a first calculation unit configured to calculate, for each pixel for distance calculation, distance information on the pixel on the basis of a local area in the first image including the pixel and a local area in the second image corresponding to the local area in the first image … a third calculation unit configured to calculate a ratio of the number of local areas for which the non-similarity is determined to be equal to or higher than the threshold in a predetermined area to the number of local areas in the predetermined area.” Komatsu teaches a calculation unit that focuses on the non-similarity of the local areas of the first and second images, and determining the pixel distance between those specific regions.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the calculation unit as taught by Komatsu with the system of Choe in order to be able to calculate and determine the pixel distance between multiple images.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Choe and in view of Kimata (US 8363985 B2) and in further view of Hahn (US 8457424 B2).
As per claim 10, Choe alone does not explicitly teach the claimed limitations.
However, Choe in combination with Kimata teaches the claimed:
10. The method of claim 1, wherein at least one of determining that the first image and second image satisfy the first similarity criterion or determining that the first image and second image satisfy the second similarity criterion comprises:
applying an interpolator to the first image and the second image to generate a test interpolated image between the first image and the second image; and
(Kimata (col 11, line 45-50): “an image interpolator 304 for generating image information at a pixel position in image E equal to each pixel position in image D”. Kimata teaches the image interpolator to generate image information.
Hahn (col 5, line 34-38): “This image can be an original image or an intermediate image to be interpolated between original images of the image sequence. The image content of an image block of such an intermediate image can be interpolated using the motion vector assigned to the image block”. Hahn teaches the image that can be an intermediate image to be interpolated between the two images.)
determining that the test interpolated image differs from the first image and the second image by less than a threshold amount.
(Kimata (col 11, line 52-56): “a difference generator 306 for computing a difference between image information at each pixel position in image A and image information at the same pixel position (as the pixel position in image A) in image F”.
Kimata (col 12, line 1-3): “The image selector 307 predetermines a threshold for the difference, and compares the threshold with the difference obtained by the difference generator 306.” Kimata teaches the difference generator that can compute a difference between the interpolated image and the first and second images. The image selector then, with a predetermined threshold, compares the threshold with the difference from the difference generator and is able to determine whether it differs less than a threshold.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the image interpolator, difference generator, and image selector as taught by Kimata with the system of Choe in order to be able to generate image information and data at specific pixel positions, compute differences between image information at pixel positions, and compare the difference values with a threshold value. Also to use the intermediate image as taught by Hahn with the system of Choe in order to generate images to be interpolated between two images.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Choe in view of Kimata and in view of Hahn, and in further view of Diggins (US 8810726 B2).
As per claim 15, Choe, Kimata, and Hahn alone do not explicitly teach the claimed limitations.
However, Choe, Kimata, and Hahn in combination with Diggins teaches the claimed:
15. The method of claim 10, wherein applying the interpolator to the first image and the second image to generate the test interpolated image between the first image and the second image includes generating a confidence output from the interpolator, and
(Diggins (col 2, line 17-20): “Where the temporal interpolation is motion compensated using motion vectors and in which the said temporal interpolation confidence measure may be derived from confidence in the motion vectors.”)
wherein determining that the test interpolated image differs from the first image and the second image by less than a threshold amount comprises determining that the confidence output exceeds a confidence threshold.
(Diggins (col 7, line 7-14): “At step (32) the buffer level is tested against a high threshold to see if is high enough to support expected periods of low confidence. If the level is below the high threshold, the confidence is tested at step (33), and if it is high, the rate of interpolation of fields is increased higher than the output field rate at step (34). If the confidence was low at step (33) the system waits for high confidence before proceeding to step (34).”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the confidence measure as taught by Diggins with the system of Choe as modified by Kimata, and Hahn in order to determine the reliability or accuracy of the interpolated image.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Choe and in view of Gong (US 20250166208 A1).
As per claim 18, Choe alone does not explicitly teach the claimed limitations.
However, Choe in combination with Gong teaches the claimed:
18. The method of claim 17, wherein at least one of determining that the first image and second image satisfy the first similarity criterion or determining that the first image and second image satisfy the second similarity criterion comprises:
determining that an optical flow consistency between the first optical flow pattern and the second optical flow pattern exceeds a threshold value.
(Gong [0007]: “calculating a first offset vector between the first optical flow of the first image block currently to be detected and the second optical flow of the second image block in the second video frame and corresponding to the first optical flow, and comparing the first offset vector with a first threshold preset; in response to the first offset vector being greater than the first threshold …”.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the optical flow as taught by Gong with the system of Choe in order to estimate and determine the movement of pixels between images to understand motion within a scene.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSHUA SUO whose telephone number is (571) 272-8387. The examiner can normally be reached Mon-Fri 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached on (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSHUA SUO/Examiner, Art Unit 2616
/DANIEL F HAJNIK/Supervisory Patent Examiner, Art Unit 2616
