DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
1. The disclosure is objected to because of the following informalities:
On page 14, line 6, "search space 325" should read "search space 322"
Appropriate correction is required.
Claim Rejections - 35 USC § 103
2. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3. Claims 1-3 and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (EP-1295483-B1, hereinafter "Kim") in view of Choudhury et al. (US-2018/0082429-A1, hereinafter "Choudhury").
4. As per claim 1, Kim discloses: A method of performing motion estimation for spatially arranged data in a data processor, the spatially arranged data having been previously compressed using a compression algorithm [[that generates compression meta data representing the spatially arranged data,]] the method comprising:
for a first region of the spatially arranged data: (Kim, page 4, [0019], “In contrast, an embodiment of a method of performing motion estimation in accordance with the present invention reduces computational complexity and enhances compression performance both in terms of the quality of the image produced and the compression efficiency that results, when compared, for example, with the logarithmic search method. ... Median motion vector components are determined from a set of neighboring macroblocks that include motion vectors.” and page 6, [0026], “An aspect of this approach is the observation that, typically, spatial correlation is present for the motion vector field among neighboring macroblocks.” and page 8, [0037], “For embodiments that are at least partially implemented in software, ... such as by a computing platform, such as a PC or other computing device, so that the system is capable of executing the instructions to result in motion estimation.”)
[[obtaining compression meta data for the first region, the compression metadata comprising smoothness information indicative of a smoothness of the first region;]] and
determining a search space size (Kim, page 6, [0026], “An aspect of this approach is the observation that, typically, spatial correlation is present for the motion vector field among neighboring macroblocks. Therefore, an efficient approach to searching may include using a smaller search window centered at PX, PY.”) based on the obtained smoothness information, the search space size being a size of a search space for applying a region-matching motion estimation algorithm to the first region to match the first region with a corresponding region in reference spatially arranged data within the search space. (Kim, page 6, [0026], “An aspect of this approach is the observation that, typically, spatial correlation is present for the motion vector field among neighboring macroblocks. Therefore, an efficient approach to searching may include using a smaller search window centered at PX, PY. Furthermore, here, by calculating the median values separately for each vector component, the opportunity for additional information is present. In particular, if the median motion vector components come from the same macroblock, this may indicate that the motion vector field is relatively 'smooth,' meaning, in this context, that there is relatively little variation between motion vectors that are located in relatively close spatial proximity. Therefore, for this embodiment, the search window is limited to nine points centered around PX, PY. However, alternatively, if the median motion vector components, here horizontal and vertical, respectively, come from different macroblocks, this may indicate that the motion field is more complicated and, therefore, the search window is increased to 25 points, although, again, centered at PX, PY.”)
5. Kim doesn't explicitly disclose but Choudhury discloses: [[A method of performing motion estimation for spatially arranged data in a data processor, the spatially arranged data having been previously compressed using a compression algorithm]] that generates compression meta data representing the spatially arranged data, (Choudhury, [0022], “Based at least in part on the one or more motion characteristics related to the one or more images, a motion characteristics metadata portion is determined. ... The one or more images are encoded into a video stream. The motion characteristics metadata portion is encoded into the video stream as a part of image metadata.” and [0090], “For example, the motion characteristics metadata may comprise a FRC data field or flag per image, per scene, per GOP, etc., to indicate one or more of: a random motion type, a smooth motion type, a panning motion type (which is considered as a smooth motion type), a random translational motion type, a smooth translational motion type, a random rotational motion type, a smooth rotational motion type, etc.” and [0035], “For example, in operational scenarios in which relatively smooth motions (including but not limited to relatively static or stationary scenes/images) are detected from two or more adjacent images, based on motion statistics indicating the relatively smooth motions ...” and [0002], “Image interpolation, which computes a set of plausible interpolated images using two or more adjacent images, has varied applications including but not limited to frame rate conversion (FRC) ...”; Examiner’s note: The metadata compression is through the encoding process with the video stream.) [[the method comprising:
for a first region of the spatially arranged data:]]
obtaining compression meta data for the first region, the compression metadata comprising smoothness information indicative of a smoothness of the first region; (Choudhury, [0022], “Based at least in part on the one or more motion characteristics related to the one or more images, a motion characteristics metadata portion is determined. ... The one or more images are encoded into a video stream. The motion characteristics metadata portion is encoded into the video stream as a part of image metadata.” and [0090], “For example, the motion characteristics metadata may comprise a FRC data field or flag per image, per scene, per GOP, etc., to indicate one or more of: a random motion type, a smooth motion type, a panning motion type (which is considered as a smooth motion type), a random translational motion type, a smooth translational motion type, a random rotational motion type, a smooth rotational motion type, etc.” and [0035], “For example, in operational scenarios in which relatively smooth motions (including but not limited to relatively static or stationary scenes/images) are detected from two or more adjacent images, based on motion statistics indicating the relatively smooth motions ...” and [0002], “Image interpolation, which computes a set of plausible interpolated images using two or more adjacent images, has varied applications including but not limited to frame rate conversion (FRC) ...”)
6. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Kim to include the disclosure of generating compressed metadata that includes smoothness information of the first region, of Choudhury. The motivation for this modification could have been to allow the motion estimation and/or encoding process to analyze the metadata and determine the best image transformation based on the metadata. For instance, based on the motion smoothness, the encoder could choose a more efficient encoding scheme.
7. As per claim 2, Kim in view of Choudhury discloses: The method of claim 1, further comprising determining whether the first region is smooth based on the smoothness information of the first region, wherein, when it is determined that the first region is smooth, setting a reduced search space as the search space size; or (Kim, page 6, [0026], “An aspect of this approach is the observation that, typically, spatial correlation is present for the motion vector field among neighboring macroblocks. Therefore, an efficient approach to searching may include using a smaller search window centered at PX, PY. Furthermore, here, by calculating the median values separately for each vector component, the opportunity for additional information is present. In particular, if the median motion vector components come from the same macroblock, this may indicate that the motion vector field is relatively 'smooth,' meaning, in this context, that there is relatively little variation between motion vectors that are located in relatively close spatial proximity. Therefore, for this embodiment, the search window is limited to nine points centered around PX, PY.”)
when it is determined that the first region is not smooth, setting an extended search space as the search space size. (Kim, page 6, [0026], “However, alternatively, if the median motion vector components, here horizontal and vertical, respectively, come from different macroblocks, this may indicate that the motion field is more complicated and, therefore, the search window is increased to 25 points, although, again, centered at PX, PY.”; Examiner’s note: As Kim discloses, if the median motion vector components come from different macroblocks, then the motion vector field is “not smooth” as previous defined in [0026] and the search window size is increased to 25 points.)
8. As per claim 3, Kim in view of Choudhury discloses: The method of claim 2, wherein the extended search space is an area within the reference spatially arranged data comprising a plurality of regions adjacent a region at a position corresponding to a position of the first region in the spatially arranged data, (Kim, Fig. 1; page 6, [0026], “However, alternatively, if the median motion vector components, here horizontal and vertical, respectively, come from different macroblocks, this may indicate that the motion field is more complicated and, therefore, the search window is increased to 25 points, although, again, centered at PX, PY.” and page 5, [0024], “In this particular embodiment, to reduce the number of bits employed for motion vector coding, motion vector components, horizontal and vertical in this embodiment, are coded differentially by using a spatial neighborhood of three macroblocks, each of the macroblocks having a motion vector, as illustrated in FIG. 1. It is also noted that this neighborhood signal information has already been transmitted in this embodiment.” and page 5, [0025], “In this embodiment, the motion vector coding is performed independently for the horizontal and vertical components. For each component, in this embodiment, the median value of the three candidates for a component is computed as follows:
PX = Median(MV1x, MV2x, MV3x)
PY = Median(MV1y, MV2y, MV3y)”;
Examiner’s note: As disclosed by Kim, the motion vector coding of PX and PY is based on a spatial neighborhood of three macroblocks. The extended search window of 25 points then surrounds a point centered on PX, PY.) and the reduced search space is a reduced area comprising a portion of the plurality of regions of the extended search space. (Kim, page 6, [0026], “An aspect of this approach is the observation that, typically, spatial correlation is present for the motion vector field among neighboring macroblocks. Therefore, an efficient approach to searching may include using a smaller search window centered at PX, PY. Furthermore, here, by calculating the median values separately for each vector component, the opportunity for additional information is present. In particular, if the median motion vector components come from the same macroblock, this may indicate that the motion vector field is relatively 'smooth,' meaning, in this context, that there is relatively little variation between motion vectors that are located in relatively close spatial proximity. Therefore, for this embodiment, the search window is limited to nine points centered around PX, PY.”; Examiner’s note: Kim discloses that the search window is centered around point PX, PY. Since the reduced search space is 9 points surrounding PX, PY and the extended search space is 25 points around the same PX, PY point, the 9 point reduced search space would be contained within the 25 point extended search space window.)
9. As per claim 16, Kim in view of Choudhury discloses: The method of claim 1, wherein applying the region-matching motion estimation algorithm comprises comparing the first region of the spatially arranged data with data elements within the search space in the reference spatially arranged data to determine a most closely matching region of the reference spatially arranged data within the search space. (Kim, page 4, [0018], “As previously indicated, a popular approach is based on a logarithmic search. In this approach, as previously described, instead of searching all of the search points within a search window, initial points are searched which are apart from each other by a quarter of the search window size. After finding a point or pixel location of the nine points which gives the least sum of absolute difference (SAD) value or some other measure, the approach then considers or checks eight additional points that are centered about that point by reducing the distance between search points by half. This approach continues until the distance between two search points becomes one pixel apart. As a result, 33 search points are checked in comparison with 1,024 for a full motion search.” and page 4, [0019], “In contrast, an embodiment of a method of performing motion estimation in accordance with the present invention reduces computational complexity and enhances compression performance both in terms of the quality of the image produced and the compression efficiency that results, when compared, for example, with the logarithmic search method. In this particular embodiment, on the transmitting side of a communications channel, for example, a method of performing motion estimation includes the following. Median motion vector components are determined from a set of neighboring macroblocks that include motion vectors.” and page 4, [0020], “Next, a window of a predetermined size and shape around a pixel location associated with the determined median motion vector components is searched. More particularly, the median motion vector components are applied to the macroblock to produce a pixel location and that pixel location is the center of the search window, for this particular embodiment. In addition to searching a window of a predetermined size and shape, a pixel location associated with a motion vector having a zero value for all of its components is also search or checked. Therefore, the pixel locations of the window and a pixel location associated with a zero motion vector are checked or searched to determine, of these, which pixel locations produce the closest match with the particular macroblock to which this technique is being applied.”)
10. Claim 17 is similar in scope to claim 1 except for a different limitation that Kim in view of Choudhury discloses: A non-transitory computer readable storage medium storing software code which when executed on one or more processors performs … (Kim, page 8, [0037], “For embodiments that are at least partially implemented in software, ... such as by a computing platform, such as a PC or other computing device, so that the system is capable of executing the instructions to result in motion estimation.” and Choudhury, [0176], “In an embodiment, a non-transitory computer readable storage medium stores software instructions, which when executed by one or more processors cause performance of a method as described herein.” and Choudhury, [0149], “In an embodiment, the one or more motion characteristics comprise a motion characteristics determined using one or more motion estimation operations that are not related to optical flow.”)
11. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the non-transitory computer readable storage medium of Kim to include the disclosure of running the motion estimation method on a non-transitory computer readable storage medium, of Choudhury. The motivation for this modification could have been to execute the motion estimation method on different types of computing hardware.
12. Claim 18 is similar in scope to claim 1 except for a different limitation that Kim in view of Choudhury discloses: A graphics processor comprising: processing circuitry … the processing circuitry being configured to: (Kim, page 8, [0037], “For embodiments that are at least partially implemented in software, ... such as by a computing platform, such as a PC or other computing device, so that the system is capable of executing the instructions to result in motion estimation.” and Choudhury, [0178], “According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices. The special-purpose computing devices may be hard-wired to perform the techniques, or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs) that are persistently programmed to perform the techniques, or may include one or more general purpose hardware processors programmed to perform the techniques pursuant to program instructions in firmware, memory, other storage, or a combination.” and Choudhury, [0149], “In an embodiment, the one or more motion characteristics comprise a motion characteristics determined using one or more motion estimation operations that are not related to optical flow.”)
13. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the graphics processor of Kim to include the disclosure of running the motion estimation method on a graphics processor, of Choudhury. The motivation for this modification could have been to execute the motion estimation method on different types of computing hardware, such as a graphics processor.
14. Claim 19, which is similar in scope to dependent claims 2-3 and independent claim 18, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 1.
15. Claims 4-5 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (EP-1295483-B1, hereinafter "Kim") in view of Choudhury et al. (US-2018/0082429-A1, hereinafter "Choudhury"), and further in view of Dane et al. (US-2009/0147853-A1, hereinafter "Dane").
16. As per claim 4, Kim in view of Choudhury discloses: The method of claim 1, wherein the smoothness information of the first region [[comprises a compressed data size of the first region.]] (See rejection for claim 1.)
17. Kim in view of Choudhury doesn't explicitly disclose but Dane discloses: [[The method of claim 1, wherein the smoothness information of the first region]] comprises a compressed data size of the first region. (Dane, [0217], “In some cases, a threshold used to classify a motion vector as a small motion vector may be fixed or adjustable based on format size. As discussed above, a small motion vector may be a nonzero motion vector that has a value below a particular threshold. In some implementations, the threshold used to determine whether a nonzero motion vector is small or not may be adjusted based on a format size of the video units being decoded and interpolated or extrapolated.” and [0082], “Encoded frames 18 may be intra-coded or inter-coded, and may be decoded to produce video content present in input frames 16. In addition, encoded frames 18 may serve as reference frames for decoding of other inter-coded frames in a video sequence, i.e., as a reference for motion estimation and motion compensation of a predicted frame. As is well known in the art of predictive coding, an encoded frame may be characterized by motion vectors that indicate displacement of blocks in the encoded frame relative to similar, corresponding blocks in a different encoded frame, which serves as a reference frame.” and [0168], “Motion analyzer 64 may analyze motion vector data from the candidate reference frames to determine whether the video scene is characterized by very little motion or significant motion.” and [0235], “In a smooth, low variance region, analysis unit 168 may use a higher quality threshold ...”; Examiner’s note: As disclosed by Dane, small/little motion (or low variance) are considered “smooth” regions.)
18. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Kim in view of Choudhury to include the disclosure that the smoothness information of the first region comprises a compressed data size of the first region, of Dane. The motivation for this modification could have been to compress a region that has smoothness information. Since regions with smooth motion tend to not have big changes between images, they have the potential to be efficiently compressed within their regions.
19. As per claim 5, Kim in view of Choudhury, and further in view of Dane discloses: The method of claim 4, further comprising determining whether the first region is smooth based on the smoothness information of the first region, wherein the first region is determined smooth when the compressed data size of the first region is below a size threshold. (Dane, [0217], “In some cases, a threshold used to classify a motion vector as a small motion vector may be fixed or adjustable based on format size. As discussed above, a small motion vector may be a nonzero motion vector that has a value below a particular threshold. In some implementations, the threshold used to determine whether a nonzero motion vector is small or not may be adjusted based on a format size of the video units being decoded and interpolated or extrapolated. …The value of the small motion vector threshold for a CIF frame may be smaller than the value of the small motion vector threshold for a VGA frame. In particular, a motion vector magnitude may be considered large in a smaller format frame but small in a larger format frame in consideration of the larger overall size of the larger format frame. Hence, in some implementations, motion analyzer 64 may adjust the small motion vector threshold value based on the format size of the video unit that is being interpolated or extrapolated. For example, the small motion vector threshold value used for smaller format frames may be less than the small motion vector threshold value used for larger format frames.” and [0082], “Encoded frames 18 may be intra-coded or inter-coded, and may be decoded to produce video content present in input frames 16. In addition, encoded frames 18 may serve as reference frames for decoding of other inter-coded frames in a video sequence, i.e., as a reference for motion estimation and motion compensation of a predicted frame. As is well known in the art of predictive coding, an encoded frame may be characterized by motion vectors that indicate displacement of blocks in the encoded frame relative to similar, corresponding blocks in a different encoded frame, which serves as a reference frame.” and [0168], “Motion analyzer 64 may analyze motion vector data from the candidate reference frames to determine whether the video scene is characterized by very little motion or significant motion.”; Examiner’s note: As disclosed by Dane, small/little motion (or low variance) are considered “smooth” regions. The motion analyzer is able to determine an appropriate threshold value based on the format size, properly determining small motion vectors (smooth regions).)
20. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 4 of Kim in view of Choudhury to include the disclosure that the first region is determined smooth when the compressed data size of the first region is below a size threshold, of Dane. The motivation for this modification could have been for the motion estimation process to quickly determine the smoothness of a region based on the image information. This could help save processing for these known smooth block regions below a threshold.
21. Claim 20, which is similar in scope to dependent claims 4-5 and independent claim 18, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claims 1 and 4-5.
22. Claims 6-13, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (EP-1295483-B1, hereinafter "Kim") in view of Choudhury et al. (US-2018/0082429-A1, hereinafter "Choudhury"), and further in view of Kumar et al. (US-2005/0094852-A1, hereinafter "Kumar").
23. As per claim 6, Kim in view of Choudhury discloses: The method of claim 1, wherein the compression algorithm is a region-based compression algorithm [[configured to generate a plurality of randomly accessible compression regions.]] (Choudhury, [0022], “Based at least in part on the one or more motion characteristics related to the one or more images, a motion characteristics metadata portion is determined. ... The one or more images are encoded into a video stream. The motion characteristics metadata portion is encoded into the video stream as a part of image metadata.” and [0104], “Additionally, optionally, or alternatively, types of motions such as random translations, random rotations, smooth translations, smooth rotations, panning motions, a combination of two or more different types of motions in images or spatial regions therein can be determined based on some or all of these parameters relating to motion characteristics in the images or the spatial regions therein.”)
24. Kim in view of Choudhury doesn't explicitly disclose but Kumar discloses: [[The method of claim 1, wherein the compression algorithm is a region-based compression algorithm]] configured to generate a plurality of randomly accessible compression regions. (Kumar, [0025], “In the preferred embodiment, a Random Sample Consensus (RANSAC) algorithm based model fitting to motion vectors of randomly selected high-translation blocks is used. This makes the method robust to outliers. Robustness to outliers can also be achieved by several variants of RANSAC. The motion vector of blocks is measured using phase correlation, which provides sub-pixel accuracy without significant computational overhead.” and [0022], “The invention is directed to methods, devices and systems for encoding and/or processing image, e.g. video, data. Embodiments of the invention provide a fast and robust global motion estimation algorithm based on two-stage coarse-to-fine refinement strategy, which is capable of measuring large motions.” and [0005], “Global motion estimation may therefore be used (by itself or in combination with a block-based motion estimation algorithm) to accomplish video compression, segmentation, mosaicing, format conversion, image registration, camera stabilization and other similar image handling tasks and image manipulation effects.”)
25. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Kim in view of Choudhury to include the disclosure of utilizing a region-based compression algorithm configured to generate a plurality of randomly accessible compression regions, of Kumar. The motivation for this modification could have been to create a more robust compression process that is able to handle a lot more motion types and potentially larger motion regions.
26. As per claim 7, Kim in view of Choudhury, and further in view of Kumar discloses: The method of claim 6, wherein the smoothness information comprises region-specific data value range information indicative of a range of data values for a region of the spatially arranged data. (Kim, page 8, [0033], “The range values, Rx and Ry, provide an indication about how the motion vector field may be changing in these directions. Therefore, if the change is relatively large, then a larger window in the direction is searched; however, if the change is relatively small, then a smaller window in that direction is searched.” and page 6, [0026], “In particular, if the median motion vector components come from the same macroblock, this may indicate that the motion vector field is relatively 'smooth,' meaning, in this context, that there is relatively little variation between motion vectors that are located in relatively close spatial proximity.”)
27. As per claim 8, Kim in view of Choudhury, and further in view of Kumar discloses: The method of claim 7, further comprising determining whether the first region is smooth based on the smoothness information of the first region, wherein the first region is determined smooth when the region-specific data value range information of the first region is below a data value range threshold. (Choudhury, [0035], “In some embodiments, the relatively smooth motions may be measured or determined so when motion vectors computed from adjacent images in the input image content have similar, relatively uniform, and/or converging magnitudes and directions with variances that do not exceed corresponding random motion thresholds.” and [0061], “In some embodiments, in response to determining that the total number of components/objects in the optical flow field of the adjacent images does not exceed the specific component number threshold, the display system avoids performing statistical analyses on the motion vectors represented in the optical flow field and directly determines that the adjacent images are to be classified as not containing random motions but rather are to be classified as containing smooth (or non-random) motions.”)
28. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 7 of Kim in view of Kumar to include the disclosure of that the first region is determined smooth when the region-specific data value range information of the first region is below a data value range threshold, of Choudhury. The motivation for this modification could have been for the motion estimation process to quickly determine the smoothness of a region based on the image information. This could help save processing for these known smooth block regions below a threshold.
29. As per claim 9, Kim in view of Choudhury, and further in view of Kumar discloses: The method of claim 6, wherein the compression meta data further comprises position information indicative of a position of a region within the spatially arranged data. (Choudhury, [0091], “For example, the motion characteristics metadata may be used to indicate that there are primarily translational motions in an image or a spatial region thereof, in a scene, etc. Additionally, optionally, or alternatively, the motion characteristics metadata may be used to indicate that there are two or more layer or spatial regions of different types of motions in an image or a spatial region thereof, in a scene, etc.”)
30. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 6 of Kim in view of Kumar to include the disclosure of that the compression metadata further comprises position information indicative of a position of a region within the spatially arranged data, of Choudhury. The motivation for this modification could have been to provide additional data about a compressed region, specifically its position. This additional information could be provided into a motion estimation and/or encoding process to analyze the metadata and determine the best image transformation based on the metadata.
31. As per claim 10, Kim in view of Choudhury, and further in view of Kumar discloses: The method of claim 9, wherein obtaining compression meta data for the first region comprises obtaining smoothness information of the first region based on the position information of the first region. (Choudhury, [0091], “For example, the motion characteristics metadata may be used to indicate that there are primarily translational motions in an image or a spatial region thereof, in a scene, etc. Additionally, optionally, or alternatively, the motion characteristics metadata may be used to indicate that there are two or more layer or spatial regions of different types of motions in an image or a spatial region thereof, in a scene, etc.” and [0090], “For example, the motion characteristics metadata may comprise a FRC data field or flag per image, per scene, per GOP, etc., to indicate one or more of: a random motion type, a smooth motion type, a panning motion type (which is considered as a smooth motion type), a random translational motion type, a smooth translational motion type, a random rotational motion type, a smooth rotational motion type, etc.” and [0104], “Additionally, optionally, or alternatively, types of motions such as random translations, random rotations, smooth translations, smooth rotations, panning motions, a combination of two or more different types of motions in images or spatial regions therein can be determined based on some or all of these parameters relating to motion characteristics in the images or the spatial regions therein.”)
32. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 9 of Kim in view of Kumar to include the disclosure of obtaining compression metadata for the first region comprises obtaining smoothness information of the first region based on the position information of the first region, of Choudhury. The motivation for this modification could have been for the motion estimation process to quickly determine the smoothness of a region based on the image information. This could help save processing for these known smooth block regions below a threshold.
33. As per claim 11, Kim in view of Choudhury, and further in view of Kumar discloses: The method of claim 6, the method further comprising, when the first region is within a single compression region, obtaining smoothness information for the first region based on smoothness information of the single compression region using position information of the single compression region. (Choudhury, [0090], “For example, the motion characteristics metadata may comprise a FRC data field or flag per image, per scene, per GOP, etc., to indicate one or more of: a random motion type, a smooth motion type, a panning motion type (which is considered as a smooth motion type), a random translational motion type, a smooth translational motion type, a random rotational motion type, a smooth rotational motion type, etc.” and [0091], “For example, the motion characteristics metadata may be used to indicate that there are primarily translational motions in an image or a spatial region thereof, in a scene, etc. Additionally, optionally, or alternatively, the motion characteristics metadata may be used to indicate that there are two or more layer or spatial regions of different types of motions in an image or a spatial region thereof, in a scene, etc.” and [0100]-[0101], “For example, a display system as described herein may be configured to partition/segment an image (or two or more adjacent images) into one or more spatial regions based on respective motion characteristics in the one or more spatial regions of the image (or the two or more adjacent images). In some embodiments, the one or more spatial regions collectively span an entire image. In some embodiments, none of the one or more spatial regions may be non-overlapping. In some embodiments, at least two of the one or more spatial regions may overlap with each other. The different motion characteristics in the different spatial regions may be determined with an optical flow field or motion vectors therein as generated from two or more adjacent images including but not necessarily limited to the (current) image. In an example, relatively high motions may be detected in a first portion of the image, whereas relatively low motions may be detected in a second portion of the image. In another example, relatively random motions may be detected in a third portion of the image, whereas relatively smooth motions may be detected in a fourth portion of the image.” and [0104], “Additionally, optionally, or alternatively, types of motions such as random translations, random rotations, smooth translations, smooth rotations, panning motions, a combination of two or more different types of motions in images or spatial regions therein can be determined based on some or all of these parameters relating to motion characteristics in the images or the spatial regions therein.” and [0022], “Based at least in part on the one or more motion characteristics related to the one or more images, a motion characteristics metadata portion is determined. ... The one or more images are encoded into a video stream. The motion characteristics metadata portion is encoded into the video stream as a part of image metadata.”)
34. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 6 of Kim in view of Kumar to include the disclosure of obtaining smoothness information for the first region based on smoothness information of the single compression region using position information of the single compression region, of Choudhury. The motivation for this modification could have been for the motion estimation process to quickly determine the smoothness of a region based on the image information. This could help save processing for these known smooth block regions below a threshold.
35. As per claim 12, Kim in view of Choudhury, and further in view of Kumar discloses: The method of claim 11, further comprising, when the first region overlaps a first compression region and a second compression region, obtaining smoothness information for the first region by comparing smoothness information of the first compression region using position information of the first compression region and smoothness information of the second compression region using position information of the second compression region. (Choudhury, [0100]-[0101], “In some embodiments, the one or more spatial regions collectively span an entire image. In some embodiments, none of the one or more spatial regions may be non-overlapping. In some embodiments, at least two of the one or more spatial regions may overlap with each other. The different motion characteristics in the different spatial regions may be determined with an optical flow field or motion vectors therein as generated from two or more adjacent images including but not necessarily limited to the (current) image.” and [0155], “In an embodiment, the video encoder is further configured to perform: comparing at least one motion characteristic in the one or more motion characteristics of the one or more images with a variance threshold; based on results of comparing the at least one motion characteristic with the variance threshold, determining whether the image content visually depicted in the one or more images comprises relatively random motions; etc.” and [0090], “For example, the motion characteristics metadata may comprise a FRC data field or flag per image, per scene, per GOP, etc., to indicate one or more of: a random motion type, a smooth motion type, a panning motion type (which is considered as a smooth motion type), a random translational motion type, a smooth translational motion type, a random rotational motion type, a smooth rotational motion type, etc.” and [0104], “Additionally, optionally, or alternatively, types of motions such as random translations, random rotations, smooth translations, smooth rotations, panning motions, a combination of two or more different types of motions in images or spatial regions therein can be determined based on some or all of these parameters relating to motion characteristics in the images or the spatial regions therein.” and [0022], “Based at least in part on the one or more motion characteristics related to the one or more images, a motion characteristics metadata portion is determined. ... The one or more images are encoded into a video stream. The motion characteristics metadata portion is encoded into the video stream as a part of image metadata.”)
36. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 11 of Kim in view of Kumar to include the disclosure of obtaining smoothness information for the first region by comparing smoothness information of the first compression region using position information of the first compression region and smoothness information of the second compression region using position information of the second compression region, of Choudhury. The motivation for this modification could have been for the motion estimation process to quickly determine the smoothness of a region by comparing two different regions and their smoothness. This could help save processing for these known smooth block regions below a threshold.
37. As per claim 13, Kim in view of Choudhury, and further in view of Kumar discloses: The method of claim 12, further comprising determining whether the first region is smooth based on the smoothness information of the first region, wherein the first region is determined to be smooth when a difference between the smoothness information of the first compression region and the smoothness information of the second compression region is below a smoothness threshold. (Choudhury, [0155], “In an embodiment, the video encoder is further configured to perform: comparing at least one motion characteristic in the one or more motion characteristics of the one or more images with a variance threshold; based on results of comparing the at least one motion characteristic with the variance threshold, determining whether the image content visually depicted in the one or more images comprises relatively random motions; etc.” and [0090], “For example, the motion characteristics metadata may comprise a FRC data field or flag per image, per scene, per GOP, etc., to indicate one or more of: a random motion type, a smooth motion type, a panning motion type (which is considered as a smooth motion type), a random translational motion type, a smooth translational motion type, a random rotational motion type, a smooth rotational motion type, etc.” and [0104], “Additionally, optionally, or alternatively, types of motions such as random translations, random rotations, smooth translations, smooth rotations, panning motions, a combination of two or more different types of motions in images or spatial regions therein can be determined based on some or all of these parameters relating to motion characteristics in the images or the spatial regions therein.” and [0022], “Based at least in part on the one or more motion characteristics related to the one or more images, a motion characteristics metadata portion is determined. ... The one or more images are encoded into a video stream. The motion characteristics metadata portion is encoded into the video stream as a part of image metadata.”)
38. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 12 of Kim in view of Kumar to include the disclosure of the first region is determined to be smooth when a difference between the smoothness information of the first compression region and the smoothness information of the second compression region is below a smoothness threshold, of Choudhury. The motivation for this modification could have been for the motion estimation process to quickly determine the smoothness of a region by comparing two different regions and their smoothness (specifically, below a specified threshold). This could help save processing for these known smooth block regions below a threshold.
39. Claim 21, which is similar in scope to dependent claims 6-8 and independent claim 18, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claims 1 and 6-8.
40. Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (EP-1295483-B1, hereinafter "Kim") in view of Choudhury et al. (US-2018/0082429-A1, hereinafter "Choudhury"), further in view of Kumar et al. (US-2005/0094852-A1, hereinafter "Kumar"), and further in view of He et al. (US-2014/0286433-A1, hereinafter "He").
41. As per claim 14, Kim in view of Choudhury, and further in view of Kumar discloses: The method of claim 11, wherein the region-based compression algorithm [[generates a hierarchical representation representing the plurality of compression regions, each compression region being sub-divided into a plurality of sub-regions, and the position information comprises a position of a sub-region respective of a compression region within the hierarchical representation.]] (Choudhury, [0022], “Based at least in part on the one or more motion characteristics related to the one or more images, a motion characteristics metadata portion is determined. ... The one or more images are encoded into a video stream. The motion characteristics metadata portion is encoded into the video stream as a part of image metadata.” and [0104], “Additionally, optionally, or alternatively, types of motions such as random translations, random rotations, smooth translations, smooth rotations, panning motions, a combination of two or more different types of motions in images or spatial regions therein can be determined based on some or all of these parameters relating to motion characteristics in the images or the spatial regions therein.”)
42. Kim in view of Choudhury, and further in view of Kumar doesn't explicitly disclose but He discloses: [[The method of claim 11, wherein the region-based compression algorithm]] generates a hierarchical representation representing the plurality of compression regions, each compression region being sub-divided into a plurality of sub-regions, and the position information comprises a position of a sub-region respective of a compression region within the hierarchical representation. (He, Fig. 5; [0022], “Motion information is utilized in video processing and compression. The present disclosure describes hierarchical motion estimation (HME) methods and related devices and systems that can provide reliable motion information for motion-related applications such as, by way of example and not of limitation, deinterlacing, denoising, super resolution, object tracking, and compression.” and He, [0081], “A co-located region from a higher h-layer or h-layers can be utilized to generate motion vectors for the current region or block Xt. The mapping motion vector of region Xt may be from the motion vector of the same region at a different h-layer as indicated by B4t-1.” and He, [0099], “As an example, one may initially conduct a spatial analysis or examine how values at co-located regions may have been changed from one h-layer to the next. Another exemplary criterion for consideration includes a value of the motion vectors (e.g., if all motion vectors are exactly zero, or maybe even close to zero, this suggests stationary status).” and He, [0097], “One may consider the relationship of a co-located block to its neighborhood, and use the resulting information to project or predict distortion behavior pattern for the current block. For example, the resulting information can be used to refine or adjust thresholding parameters for the current block.” and He, [0068], “The exemplary application of HME considers at each h-layer (h-layer-1 (1120) in the example shown in FIG. 11) blocks that are of a certain larger partition size, which are later subdivided to a smaller partition size when moving to the next h-layer (1110). This means that before subdivision, motion for multiple adjacent partitions was estimated but as a single group/partition. The refinement at the next h-layer (1110) is commonly constrained around a smaller search window, making the search more correlated. The derived MV predictor can be generated with any existing predictors by means of, for example, some mathematic operation such as median filtering or weighted average.” and He, [0058], “The present disclosure describes motion vector (MV) prediction in HME, HME based fast motion search, and how HME information can be utilized." and He, [0035], “As used in this disclosure, the term “hierarchical layer” or “h-layer” refers to a full set, a superset, or a subset of an input picture of video information for use in HME processes.”)
43. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 11 of Kim in view of Choudhury, and further in view of Kumar to include the disclosure of that the region-based compression algorithm generates a hierarchical representation representing the plurality of compression regions, each compression region being sub-divided into a plurality of sub-regions, and the position information comprises a position of a sub-region respective of a compression region within the hierarchical representation, of He. The motivation for this modification could have been to generate a structural hierarchy for image(s) to be able to quickly traverse different parts of the image(s). This can be especially useful for region comparison by traversing the hierarchy quickly to obtain two blocks for motion estimation comparison.
44. As per claim 15, Kim in view of Choudhury, further in view of Kumar, and further in view of He discloses: The method of claim 14, further comprising, when the first region overlaps a first sub-region of a compression region and a second sub-region of the compression region, determining that the first sub-region and the second sub-region belong to the same compression region within the hierarchical representation using position information of the first sub-region and the position information of the second sub-region, and obtaining smoothness information for the first region based on smoothness information of said same compression region. (He, Fig. 5; [0022], “Motion information is utilized in video processing and compression. The present disclosure describes hierarchical motion estimation (HME) methods and related devices and systems that can provide reliable motion information for motion-related applications such as, by way of example and not of limitation, deinterlacing, denoising, super resolution, object tracking, and compression.” and [0068], “The exemplary application of HME considers at each h-layer (h-layer-1 (1120) in the example shown in FIG. 11) blocks that are of a certain larger partition size, which are later subdivided to a smaller partition size when moving to the next h-layer (1110). This means that before subdivision, motion for multiple adjacent partitions was estimated but as a single group/partition. The refinement at the next h-layer (1110) is commonly constrained around a smaller search window, making the search more correlated. The derived MV predictor can be generated with any existing predictors by means of, for example, some mathematic operation such as median filtering or weighted average.” and [0058], “The present disclosure describes motion vector (MV) prediction in HME, HME based fast motion search, and how HME information can be utilized." and [0035], “As used in this disclosure, the term “hierarchical layer” or “h-layer” refers to a full set, a superset, or a subset of an input picture of video information for use in HME processes.” and Choudhury, [0100]-[0101], “In some embodiments, the one or more spatial regions collectively span an entire image. In some embodiments, none of the one or more spatial regions may be non-overlapping. In some embodiments, at least two of the one or more spatial regions may overlap with each other. The different motion characteristics in the different spatial regions may be determined with an optical flow field or motion vectors therein as generated from two or more adjacent images including but not necessarily limited to the (current) image.” and Choudhury, [0090], “For example, the motion characteristics metadata may comprise a FRC data field or flag per image, per scene, per GOP, etc., to indicate one or more of: a random motion type, a smooth motion type, a panning motion type (which is considered as a smooth motion type), a random translational motion type, a smooth translational motion type, a random rotational motion type, a smooth rotational motion type, etc.” and Choudhury, [0022], “Based at least in part on the one or more motion characteristics related to the one or more images, a motion characteristics metadata portion is determined. ... The one or more images are encoded into a video stream. The motion characteristics metadata portion is encoded into the video stream as a part of image metadata.”)
45. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 14 of Kim in view of Choudhury, and further in view of Kumar to include the disclosure of when the first region overlaps a first sub-region of a compression region and a second sub-region of the compression region, determining that the first sub-region and the second sub-region belong to the same compression region within the hierarchical representation using position information of the first sub-region and the position information of the second sub-region, and obtaining smoothness information for the first region based on smoothness information of said same compression region, of He. The motivation for this modification could have been to quickly determine whether two sub-regions are within the same hierarchical region. By identifying these sub-regions overlap, this can save some processing as information related to one sub-region is likely similar or identical to the information in the other sub-region. For instance, this could indicate smoothness properties of both sub-regions as being identical or near identical.
Conclusion
46. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW CLOTHIER whose telephone number is (571)272-4667. The examiner can normally be reached Mon-Fri 8:00am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW CLOTHIER/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights