DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO internet Web site contains terminal disclaimer forms which may be used. Please visit http://www.uspto.gov/forms/. The filing date of the application will determine what form should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1, 6 and 11 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1 and 10 of Patent No. 12,229,970 B2. Although the conflicting claims are not identical, they are not patentably distinct from each other because: the instant claims are substantially similar to the claims in the conflicting patent, as shown in the following tables.
Claim 1 is rejected for obviousness-type double patenting under claim 1 of Patent No. 12,229,970 B2
Instant application claim 1
Claim 1 of Patent No. 12,229,970
A method comprising:
A method comprising:
generating one or more first motion vectors mapping first visual content displayed at one or more first pixel locations in a first reference frame of a sequence of frames to one or more pixel locations in an intermediate frame;
one or more second motion vectors mapping one or more second frames to the intermediate frame;
generating one or more second motion vectors mapping second visual content displayed at one or more second pixel locations in a second reference frame of the sequence of frames to the one or more pixel locations in the intermediate frame;
identifying an inconsistency between one or more first motion vectors mapping one or more first frames to an intermediate frame and one or more second motion vectors mapping one or more second frames to the intermediate frame;
detecting, based at least on the first visual content and the second visual content, a conflict between the one or more first motion vectors and the one or more second motion vectors at least one pixel location of the one or more pixel locations in the intermediate frame; and
comparing at least one motion vector corresponding to the one or more first frames to a reference motion vector; and computing pixel data corresponding to the intermediate frame based at least on the comparison and the inconsistency.
responsive to the detecting, generating pixel data for the at least one pixel location in the intermediate frame using at least one motion vector of the one or more first motion vectors based at least on the one or more first motion vectors having a greater similarity to a global motion vector than the one or more second motion vectors
Claim 6 is rejected for obviousness-type double patenting under claim 1 of Patent No. 12,229,970 B2
Instant application claim 6
Claim 1 of Patent No. 12,229,970
The method of claim 1,
A method comprising:
wherein the comparison includes determining a level of similarity between the at least one motion vector and the reference motion vector, and the pixel data corresponds to the one or more first motion vectors based at least on the level of similarity.
responsive to the detecting, generating pixel data for the at least one pixel location in the intermediate frame using at least one motion vector of the one or more first motion vectors based at least on the one or more first motion vectors having a greater similarity to a global motion vector than the one or more second motion vectors
Claim 11 is rejected for obviousness-type double patenting under claim 10 of Patent No. 12,229,970 B2
Instant application claim 11
Claim 1 of Patent No. 12,229,970
A system comprising:
A system comprising:
one or more processors to perform operations including:
one or more processing units to perform operations comprising:
one or more first motion vectors mapping one or more first frames to an intermediate frame and one or more second motion vectors mapping one or more second frames to the intermediate frame;
generating, based at least on an optical flow between a first reference frame and a second reference frame, a first motion vector mapping first visual content of the first reference frame to a pixel location in an intermediate frame and a second motion vector mapping second visual content of the second reference frame to the pixel location in the intermediate frame;
identifying an inconsistency between one or more first motion vectors mapping one or more first frames to an intermediate frame and one or more second motion vectors mapping one or more second frames to the intermediate frame;
determining, based at least on the first visual content and the second visual content, a conflict between the first motion vector and the second motion vector at the pixel location; and
comparing at least one motion vector corresponding to the one or more first frames to a reference motion vector corresponding to the one or more first frames; and computing pixel data corresponding to at least one location of the intermediate frame based at least on the comparison and the inconsistency
generating pixel data corresponding to the pixel location using the first motion vector based at least on the first motion vector having a greater similarity to a global motion vector than the second motion vector.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-13, 15-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Barone (U.S. Patent Application Publication 2007/0285430 A1) in view of Fishwick (U.S. Patent Application Publication 2016/0021386 A1).
Regarding claim 1, Barone discloses a method comprising:
identifying an inconsistency (FIGS. 3, 4 and 5; paragraphs [0087]-[0088], in order to generate the interpolated frames IF in succession, a fully rendered forward frame KFF and a backward frame KFB should be available. A backward frame KFB is a reference frame at a later time in display order, while a forward frame KFF is a reference frame at an earlier time in display order ... As its input, the motion compensation stage 237 thus receives a fully rendered frame KF, the frame interpolation index and the motion buffer information MB, and comprises a first buffer (designated a forward motion buffer MBF; paragraph [0123], the location (i,j) of the forward motion buffer MBF includes three coordinates (x0,y0,z0)i,j related to the (i,j) pixel in the interpolated frame IF that point at the forward frame KFF. Similarly the location (i,j) in the backward motion buffer MBB includes (xn+1,yn+1,zn+1)i,j coordinates related to the (i,j) pixel in the interpolated frame IF that point at the backward frame KFB) associated to a forward frame KFF, and a second buffer (designated a backward motion buffer MBB) associated to a backward frame KFB; paragraphs [0138]-[0140], the motion compensation stage 237 performs a general process for computing colors having as its inputs i) two frames KF (forward frame KFF and backward frame KFB) already rendered by the rendering stage 213, to be used as references; ... If a pixel (i,j) is not in the background, a check operation is performed to test if the motion vectors are valid or not .. After such operation if both motion vectors for the pixel (i,j) are valid, the final color is computed using four pixels that overlap the quantized position addressed on the forward frame KFF. The same applies to the backward frame KFB. A total of 8 pixels are used with weighting factors wi con i=1 . . . 8. If only one motion vector is valid ... Thus, an inconsistency between motion vectors is identified if only one motion vector is valid) between one or more first motion vectors (FIG. 6; paragraph [0106], v(x0,y0,z.0) is the notation for a generic motion vector that expresses the motion between a vertex v for the (x1,y1,z1) position at the time instant t1, related to the only interpolated frame available IF) mapping one or more first frames (Paragraph [0105], the forward frame KFF) to an intermediate frame (Paragraph [0106], interpolated frame IF) and one or more second motion vectors (Paragraph [0106], the motion between a vertex v at the position (x1,y1,z1) at the instant t1 (again related to the interpolated frame IF) and the position (x2,y2,z2) at the time instant t2) mapping one or more second frames (Paragraph [0106], the backward frame KFB) to the intermediate frame (Paragraph [0106], interpolated frame IF);
computing pixel data corresponding to the intermediate frame (Paragraph [0140], if only one motion vector is valid, this is used to compute the color of the pixel. Only 4 pixels and weighting factors are needed, related to a fully rendered frame KF).
However, Barone does not specifically disclose comparing at least one motion vector corresponding to the one or more first frames to a reference motion vector; and
computing pixel data corresponding to the intermediate frame based at least on the comparison and the inconsistency.
In additional, Fishwick discloses (Paragraph [0003], FIG. 1 shows a moving object 110 at a certain position in one frame of a video sequence, and, superimposed onto the same figure, the same object 115, at its position in the previous frame in the sequence …; paragraph [0051], FIG. 8 shows three examples, labeled A), B) and C)) comparing at least one motion vector (Paragraph [0059], example B) shows the ended vectors 850 and 860) corresponding to the one or more first frames (Paragraph [0051], , in FIG. 8, an interpolated frame 801 has an image space which is divided into a plurality of regions … The scene to which motion estimation is being applied includes objects that are generally moving from the bottom to the top of the frame. In particular, lower block 820 represents the position of an object, or part of an object, 825 in a first frame, and upper block 830 represents the position of the same object portion 835 in a second frame. The interpolated frame 801 is to be an interpolation between the first frame and the second frame) to a reference motion vector (Paragraph [0059], one set of properties associated with the remote double ended vectors 850 and 860 that may be analysed are those of the vectors themselves. Specifically, the magnitude and/or the direction of the remote vectors can be determined and compared with the magnitude and/or direction of the candidate vector 840 to ascertain whether the vectors are the same or different … a global motion pan caused by something like a camera pan. Although a small amount of variation can be expected because things do not always move by exactly the same amount (e.g. the camera pan may actually be a rotation where the exact angle and distance of the vector is dependent on the distance from the camera to the object being tracked), as the difference between the vectors increases it becomes less certain that the objects are moving in the same way); and
computing pixel data corresponding to the intermediate frame (Paragraphs [0051]-[0052], the interpolated frame 801 is to be an interpolation between the first frame and the second frame … the general principle of double ended vector motion estimation is to select a candidate region, such as block 810, on which motion estimation is to be performed, and to apply a set of rules to determine a bi-directional vector that best matches the rules in relation to other potential vectors. Specifically the aim is to identify the vector that most accurately reflects the movement of pixels through the selected block. The best match, or matches, provided by the rules are termed, in this description, the candidate vectors. There may be only a single candidate vector, or there may be multiple candidate vectors that need to be chosen between in order to avoid artefacts. The general process of double ended vector motion estimation is then repeated for each block of a given frame until the entire frame has been processed) based at least on the comparison (Paragraph [0059], a global motion pan caused by something like a camera pan. Although a small amount of variation can be expected because things do not always move by exactly the same amount (e.g. the camera pan may actually be a rotation where the exact angle and distance of the vector is dependent on the distance from the camera to the object being tracked), as the difference between the vectors increases it becomes less certain that the objects are moving in the same way) and the inconsistency (Paragraph [0060], another property associated with the remote double ended vectors 850 and 860 that may be analysed is how similar the contents of the areas of image data pointed to by either end of each vector are to each other. This will be referred to as the pixel match value of the double ended vector, as described above, and can be measured by quantities such as the sum of absolute differences (SAD) or mean square differences (MSD) … For example, a higher pixel match value may indicate a better pixel match between the two areas at either end of the vector, and so the higher the probability is that the selected candidate vector 840 is correct … If it had a poor pixel match but just happened to have the same size and direction as vector 840 then it may be treated with suspicion. This is because otherwise there is an assumption that the vector matching process works because the surrounding area knows exactly where the objects are all moving and it is only the current location that is uncertain …).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the pipelined graphic engine for generating image frames for display taught by Barone incorporate the teachings of Fishwick, and applying the motion estimation taught by Fishwick to provide the double ended vectors for motion estimation between two frames; detect a conflict between motion vectors; perform the comparison between motion vectors and global motion to generate pixel data corresponding to the intermediate frame. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Barone according to the relied-upon teachings of Fishwick to obtain the invention as specified in claim.
Regarding claim 2, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 1), and Barone disclose further comprising selecting, based at least on the comparison, a subset of one or more motion vectors from the one or more first motion vectors and the one or more second motion vectors (FIG. 6; paragraph [0106], v(x0,y0,z.0) is the notation for a generic motion vector that expresses the motion between a vertex v for the (x1,y1,z1) position at the time instant t1, related to the only interpolated frame available IF ...; paragraph [0109], by considering a vertex v of a scene triangle having (xc,yc,z.sc) as a set of three-dimensional coordinates specified during the execution time of the application ...; paragraphs [0139]-[0140], if a pixel (i,j) is not in the background, a check operation is performed to test if the motion vectors are valid or not .. After such operation if both motion vectors for the pixel (i,j) are valid, the final color is computed using four pixels that overlap the quantized position addressed on the forward frame KFF. The same applies to the backward frame KFB. A total of 8 pixels are used with weighting factors wi con i=1 . . . 8. If only one motion vector is valid .... Thus, a set of v(xc,yc,z.sc) motion vectors is selected), wherein the computing the pixel data comprises computing the pixel data using the selected subset of one or more motion vectors (Paragraph [0140], if only one motion vector is valid, this is used to compute the color of the pixel. Only 4 pixels and weighting factors are needed, related to a fully rendered frame KF).
Regarding claim 3, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 2), and Barone further disclose wherein the selecting comprises selecting at least one motion vector for the subset of motion vectors (FIG. 6; paragraph [0106], v(x0,y0,z.0) is the notation for a generic motion vector that expresses the motion between a vertex v for the (x1,y1,z1) position at the time instant t1, related to the only interpolated frame available IF ...; paragraphs [0139]-[0140], if a pixel (i,j) is not in the background, a check operation is performed to test if the motion vectors are valid or not .. After such operation if both motion vectors for the pixel (i,j) are valid, the final color is computed using four pixels that overlap the quantized position addressed on the forward frame KFF. The same applies to the backward frame KFB. A total of 8 pixels are used with weighting factors wi con i=1 . . . 8. If only one motion vector is valid .... Thus, a set of v(xc,yc,z.sc) motion vectors is selected from first motion vectors or second motion vectors) from the one or more first motion vectors (Paragraph [0106], v(x0,y0,z.0) is the notation for a generic motion vector that expresses the motion between a vertex v for the (x1,y1,z1) position at the time instant t1, related to the only interpolated frame available IF) or the one or more second motion vectors (Paragraph [0106], the motion between a vertex v at the position (x1,y1,z1) at the instant t1 (again related to the interpolated frame IF) and the position (x2,y2,z2) at the time instant t2).
Regarding claim 5, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 1), and Barone further disclose wherein the reference motion vector indicates global motion of at least one of the one or more first frames or the one or more second frames (Paragraphs [0013]-[0014], global motion is a result of the object motion and the motion (if any) of the viewpoint ... The pixels compose a frame that is a snapshot from the camera viewpoint. A typical frame-rate of 20-30 frames per second (fps) is needed to achieve a fluid motion perception by the viewer).
Regarding claim 6, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 1).
However, Barone does not specifically disclose wherein the comparison includes determining a level of similarity between the at least one motion vector and the reference motion vector, and the pixel data corresponds to the one or more first motion vectors based at least on the level of similarity.
In the similar field of endeavor, Fishwick discloses (Paragraph [0003], FIG. 1 shows a moving object 110 at a certain position in one frame of a video sequence, and, superimposed onto the same figure, the same object 115, at its position in the previous frame in the sequence …; paragraph [0051], FIG. 8 shows three examples, labeled A), B) and C). Generally, in FIG. 8, an interpolated frame 801 has an image space which is divided into a plurality of regions … The scene to which motion estimation is being applied includes objects that are generally moving from the bottom to the top of the frame. In particular, lower block 820 represents the position of an object, or part of an object, 825 in a first frame, and upper block 830 represents the position of the same object portion 835 in a second frame. The interpolated frame 801 is to be an interpolation between the first frame and the second frame) wherein the comparison includes determining a level of similarity between the at least one motion vector (Paragraph [0060], a higher pixel match value may indicate a better pixel match between the two areas at either end of the vector, and so the higher the probability is that the selected candidate vector 840 is correct) and the reference motion vector (Paragraph [0059], one set of properties associated with the remote double ended vectors 850 and 860 that may be analysed are those of the vectors themselves. Specifically, the magnitude and/or the direction of the remote vectors can be determined and compared with the magnitude and/or direction of the candidate vector 840 to ascertain whether the vectors are the same or different … a global motion pan caused by something like a camera pan. Although a small amount of variation can be expected because things do not always move by exactly the same amount (e.g. the camera pan may actually be a rotation where the exact angle and distance of the vector is dependent on the distance from the camera to the object being tracked), as the difference between the vectors increases it becomes less certain that the objects are moving in the same way), and the pixel data (Paragraph [0059], the closer the match between the vectors, the more likely it is that the selected candidate vector 840 is correct. For example, if the vector 850 and/or 860 is the same at the offset location 820 and/or 830 as the vector 840 for current block 810 then it can be assumed that the objects described by those vectors are moving in the same way …) corresponds to the one or more first motion vectors based at least on the level of similarity (Paragraph [0060], a higher pixel match value may indicate a better pixel match between the two areas at either end of the vector, and so the higher the probability is that the selected candidate vector 840 is correct) relative to a global motion vector (Paragraph [0059], one set of properties associated with the remote double ended vectors 850 and 860 that may be analysed are those of the vectors themselves. Specifically, the magnitude and/or the direction of the remote vectors can be determined and compared with the magnitude and/or direction of the candidate vector 840 to ascertain whether the vectors are the same or different … a global motion pan caused by something like a camera pan. Although a small amount of variation can be expected because things do not always move by exactly the same amount (e.g. the camera pan may actually be a rotation where the exact angle and distance of the vector is dependent on the distance from the camera to the object being tracked), as the difference between the vectors increases it becomes less certain that the objects are moving in the same way).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the pipelined graphic engine for generating image frames for display taught by Barone incorporate the teachings of Fishwick, and applying the motion estimation taught by Fishwick to provide the double ended vectors for motion estimation between two frames; use the double ended vectors for mapping each region of two frames to a region of an interpolated frame; perform the comparison between motion vectors to find the best match; then the interpolated frame is to be an interpolation between the first frame and the second frame based on the best match. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Barone according to the relied-upon teachings of Fishwick to obtain the invention as specified in claim.
Regarding claim 7, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 1).
However, Barone does not specifically disclose wherein the comparison includes evaluating one or more of:
a difference between an angle of the reference motion vector and one or more angles of the at least one motion vector; or
a difference between a magnitude of the reference motion vector and one or more magnitudes of the at least one motion vector.
In the similar field of endeavor, Fishwick discloses wherein the comparison includes evaluating one or more of:
a difference between an angle of the reference motion vector and one or more angles of the at least one motion vector (FIG. 8; example B) shows the case where the resulting remote vectors are double ended vectors 850 and 860. One set of properties associated with the remote double ended vectors 850 and 860 that may be analysed are those of the vectors themselves … a global motion pan caused by something like a camera pan. Although a small amount of variation can be expected because things do not always move by exactly the same amount (e.g. the camera pan may actually be a rotation where the exact angle and distance of the vector is dependent on the distance from the camera to the object being tracked), as the difference between the vectors increases it becomes less certain that the objects are moving in the same way); or
a difference between a magnitude of the reference motion vector and one or more magnitudes of the at least one motion vector.
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the pipelined graphic engine for generating image frames for display taught by Barone incorporate the teachings of Fishwick, and applying the motion estimation taught by Fishwick to provide the double ended vectors for motion estimation between two frames; use the double ended vectors for mapping each region of two frames to a region of an interpolated frame; perform the comparison between motion vectors to find the best match; then the interpolated frame is to be an interpolation between the first frame and the second frame based on the best match. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Barone according to the relied-upon teachings of Fishwick to obtain the invention as specified in claim.
Regarding claim 8, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 1).
However, Barone does not specifically disclose wherein the comparison includes computing a first similarity score between the one or more first motion vectors and the reference motion vector and the computing of the pixel data is based at least on comparing the first similarity score to a second similarity score between the one or more second motion vectors and the reference motion vector.
In the similar field of endeavor, Fishwick discloses (Paragraph [0071], FIG. 10 outlines an example of a method that can be applied to bias the selection of candidate vectors in situations such as that described in relation to FIGS. 8 and 9)) wherein the comparison includes computing a first similarity score between the one or more first motion vectors and the reference motion vector (Paragraph [0072], if two or more candidate vectors, C1, C2, . . . , CN, etc, are identified by the rules, then an artefact risk occurs and the best vector needs to be determined from the set of vectors. At step 1003 one of the candidate vectors, e.g., vector C1, is selected for investigation from the group. The selected candidate vector is then used to select a second region based upon the position to which one end of this vector points. In FIG. 8, the example vector 840 is shown as pointing to particular grid regions 820 and 830) and the computing of the pixel data is based at least on comparing the first similarity score to a second similarity score (Paragraph [0060], a higher pixel match value may indicate a better pixel match between the two areas at either end of the vector, and so the higher the probability is that the selected candidate vector 840 is correct) between the one or more second motion vectors and the reference motion vector (Paragraph [0074], the identified remote vector R1 is analysed by comparing its magnitude and/or direction with that of the selected candidate vector C1 at step 1005. Also, the pixel match property, or SAD value, is used. If the remote vector R1 is similar enough to the selected candidate vector C1 and the pixel match of the remote vector R1 is good enough, then at step 1006 either no bias is applied to the candidate vector C1, or a bias towards the candidate vector C1 is applied (possibly by penalising the other candidates in the candidate set)).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the pipelined graphic engine for generating image frames for display taught by Barone incorporate the teachings of Fishwick, and applying the motion estimation taught by Fishwick to provide the double ended vectors for motion estimation between two frames; use the double ended vectors for mapping each region of two frames to a region of an interpolated frame; perform the comparison between motion vectors to find the best match; then the interpolated frame is to be an interpolation between the first frame and the second frame based on the best match. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Barone according to the relied-upon teachings of Fishwick to obtain the invention as specified in claim.
Regarding claim 9, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 1).
However, Barone does not specifically disclose wherein based at least on the comparison, the pixel data does not correspond to the one or more second motion vectors.
In additional, Fishwick discloses wherein based at least on the comparison, the pixel data does not correspond to the one or more second motion vectors (Paragraphs [0071]-[0074], FIG. 10 outlines an example of a method that can be applied to bias the selection of candidate vectors in situations such as that described in relation to FIGS. 8 and 9. Firstly, at step 1001, existing double ended vector motion estimation rules are applied to a first region upon which motion estimation is being performed. These motion estimation rules may, for example, apply restrictions to a set of vectors and apply weightings based upon pixel matching or other factors to arrive at a set of candidate vectors that represent the possible true motion vectors of an interpolated frame. If only a single candidate vector is identified, as may be the case in example A), B) and C) of FIG. 8, then this candidate vector is chosen as the double ended motion vector for the block of interest as indicated at step 1002 ... At step 1003 one of the candidate vectors, e.g., vector C1, is selected for investigation from the group ... At step 1004, the existing vector analysis rules are applied to the second region to identify the best matching remote vector, R1, which represents the best match to the vector analysis rules for that region in the interpolated frame ... The identified remote vector R1 is analysed by comparing its magnitude and/or direction with that of the selected candidate vector C1 at step 1005. Also, the pixel match property, or SAD value, is used. If the remote vector R1 is similar enough to the selected candidate vector C1 and the pixel match of the remote vector R1 is good enough, then at step 1006 either no bias is applied to the candidate vector C1, or a bias towards the candidate vector C1 is applied (possibly by penalising the other candidates in the candidate set). If the remote vector R1 is similar enough to the selected candidate vector C1, but the pixel match of the remote vector 1 is poor, then no bias may be applied to the candidate vector, or a bias may be applied away from the candidate vector. If the remote vector R1 is not similar enough to the selected candidate vector C1 then a bias may be applied away from the candidate vector).
Regarding claim 10, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 1), and Barone disclose further comprising streaming, to one or more client devices (FIG. 1; paragraph [0030], the outcome of the operations performed in the stages from 101 to 113 is finally supplied to a display frame device 130; FIG. 3; paragraph [0073], the motion buffer encoder 214 computes the difference between incoming motion vectors. The difference is then coded using a variable length code (VLC) in the form of coded motion buffer information in a bitstream R4 that is sent through a transmitter block 224 on a wireless or wired channel to a remote smart display 230. A video encoder 215 encodes the fully rendered frames KF output at a full rate F2 by the 3D engine 210 and sends a bitstream R3 to a transmitter block 223 that transmits such a bitstream R3 to the smart display 230), video data corresponding to the pixel data (Paragraph [0074], considering a frame comprised of a bi-dimensional matrix of N rows by M columns of RGB colors at a reduced frame rate F2 and using a compression method adapted to compress 24 bits to 6 bits, the bitstream R3 has a rate equal to (N*M*6*F2)).
Regarding claim 11, Barone discloses a system comprising:
one or more processors to perform operations (Paragraph [0190], FIG. 18 shows how a double perspective correction module 400 as described herein can be integrated in a HW accelerated 3D engine system supported by a bus 402 and including a host CPU 404) including:
identifying an inconsistency (FIGS. 3, 4 and 5; paragraphs [0087]-[0088], in order to generate the interpolated frames IF in succession, a fully rendered forward frame KFF and a backward frame KFB should be available. A backward frame KFB is a reference frame at a later time in display order, while a forward frame KFF is a reference frame at an earlier time in display order ... As its input, the motion compensation stage 237 thus receives a fully rendered frame KF, the frame interpolation index and the motion buffer information MB, and comprises a first buffer (designated a forward motion buffer MBF; paragraph [0123], the location (i,j) of the forward motion buffer MBF includes three coordinates (x0,y0,z0)i,j related to the (i,j) pixel in the interpolated frame IF that point at the forward frame KFF. Similarly the location (i,j) in the backward motion buffer MBB includes (xn+1,yn+1,zn+1)i,j coordinates related to the (i,j) pixel in the interpolated frame IF that point at the backward frame KFB) associated to a forward frame KFF, and a second buffer (designated a backward motion buffer MBB) associated to a backward frame KFB; paragraphs [0138]-[0140], the motion compensation stage 237 performs a general process for computing colors having as its inputs i) two frames KF (forward frame KFF and backward frame KFB) already rendered by the rendering stage 213, to be used as references; ... If a pixel (i,j) is not in the background, a check operation is performed to test if the motion vectors are valid or not .. After such operation if both motion vectors for the pixel (i,j) are valid, the final color is computed using four pixels that overlap the quantized position addressed on the forward frame KFF. The same applies to the backward frame KFB. A total of 8 pixels are used with weighting factors wi con i=1 . . . 8. If only one motion vector is valid ... Thus, an inconsistency between motion vectors is identified if only one motion vector is valid) between one or more first motion vectors (FIG. 6; paragraph [0106], v(x0,y0,z.0) is the notation for a generic motion vector that expresses the motion between a vertex v for the (x1,y1,z1) position at the time instant t1, related to the only interpolated frame available IF) mapping one or more first frames (Paragraph [0105], the forward frame KFF) to an intermediate frame (Paragraph [0106], interpolated frame IF) and one or more second motion vectors (Paragraph [0106], the motion between a vertex v at the position (x1,y1,z1) at the instant t1 (again related to the interpolated frame IF) and the position (x2,y2,z2) at the time instant t2) mapping one or more second frames (Paragraph [0106], the backward frame KFB) to the intermediate frame (Paragraph [0106], interpolated frame IF);
computing pixel data corresponding to at least one location of the intermediate frame (Paragraph [0140], if only one motion vector is valid, this is used to compute the color of the pixel. Only 4 pixels and weighting factors are needed, related to a fully rendered frame KF).
However, Barone does not specifically disclose comparing at least one motion vector corresponding to the one or more first frames to a reference motion vector corresponding to the one or more first frames; and
computing pixel data corresponding to at least one location of the intermediate frame based at least on the comparison and the inconsistency.
In additional, Fishwick discloses (Paragraph [0003], FIG. 1 shows a moving object 110 at a certain position in one frame of a video sequence, and, superimposed onto the same figure, the same object 115, at its position in the previous frame in the sequence …; paragraph [0051], FIG. 8 shows three examples, labeled A), B) and C)) comparing at least one motion vector (Paragraph [0059], example B) shows the ended vectors 850 and 860) corresponding to the one or more first frames (Paragraph [0051], , in FIG. 8, an interpolated frame 801 has an image space which is divided into a plurality of regions … The scene to which motion estimation is being applied includes objects that are generally moving from the bottom to the top of the frame. In particular, lower block 820 represents the position of an object, or part of an object, 825 in a first frame, and upper block 830 represents the position of the same object portion 835 in a second frame. The interpolated frame 801 is to be an interpolation between the first frame and the second frame) to a reference motion vector corresponding to the one or more first frames (Paragraph [0059], one set of properties associated with the remote double ended vectors 850 and 860 that may be analysed are those of the vectors themselves. Specifically, the magnitude and/or the direction of the remote vectors can be determined and compared with the magnitude and/or direction of the candidate vector 840 to ascertain whether the vectors are the same or different … a global motion pan caused by something like a camera pan. Although a small amount of variation can be expected because things do not always move by exactly the same amount (e.g. the camera pan may actually be a rotation where the exact angle and distance of the vector is dependent on the distance from the camera to the object being tracked), as the difference between the vectors increases it becomes less certain that the objects are moving in the same way); and
computing pixel data corresponding to at least one location of the intermediate frame (Paragraphs [0051]-[0052], the interpolated frame 801 is to be an interpolation between the first frame and the second frame … the general principle of double ended vector motion estimation is to select a candidate region, such as block 810, on which motion estimation is to be performed, and to apply a set of rules to determine a bi-directional vector that best matches the rules in relation to other potential vectors. Specifically the aim is to identify the vector that most accurately reflects the movement of pixels through the selected block. The best match, or matches, provided by the rules are termed, in this description, the candidate vectors. There may be only a single candidate vector, or there may be multiple candidate vectors that need to be chosen between in order to avoid artefacts. The general process of double ended vector motion estimation is then repeated for each block of a given frame until the entire frame has been processed) based at least on the comparison (Paragraph [0059], a global motion pan caused by something like a camera pan. Although a small amount of variation can be expected because things do not always move by exactly the same amount (e.g. the camera pan may actually be a rotation where the exact angle and distance of the vector is dependent on the distance from the camera to the object being tracked), as the difference between the vectors increases it becomes less certain that the objects are moving in the same way) and the inconsistency (Paragraph [0060], another property associated with the remote double ended vectors 850 and 860 that may be analysed is how similar the contents of the areas of image data pointed to by either end of each vector are to each other. This will be referred to as the pixel match value of the double ended vector, as described above, and can be measured by quantities such as the sum of absolute differences (SAD) or mean square differences (MSD) … For example, a higher pixel match value may indicate a better pixel match between the two areas at either end of the vector, and so the higher the probability is that the selected candidate vector 840 is correct … If it had a poor pixel match but just happened to have the same size and direction as vector 840 then it may be treated with suspicion. This is because otherwise there is an assumption that the vector matching process works because the surrounding area knows exactly where the objects are all moving and it is only the current location that is uncertain …).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the pipelined graphic engine for generating image frames for display taught by Barone incorporate the teachings of Fishwick, and applying the motion estimation taught by Fishwick to provide the double ended vectors for motion estimation between two frames; detect a conflict between motion vectors; perform the comparison between motion vectors and global motion to generate pixel data corresponding to the intermediate frame. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Barone according to the relied-upon teachings of Fishwick to obtain the invention as specified in claim.
Regarding claim 12, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 11), and Barone wherein the operations include selecting, based at least on the comparison, a subset of one or more motion vectors from the one or more first motion vectors and the one or more second motion vectors (FIG. 6; paragraph [0106], v(x0,y0,z.0) is the notation for a generic motion vector that expresses the motion between a vertex v for the (x1,y1,z1) position at the time instant t1, related to the only interpolated frame available IF ...; paragraph [0109], by considering a vertex v of a scene triangle having (xc,yc,z.sc) as a set of three-dimensional coordinates specified during the execution time of the application ...; paragraphs [0139]-[0140], if a pixel (i,j) is not in the background, a check operation is performed to test if the motion vectors are valid or not .. After such operation if both motion vectors for the pixel (i,j) are valid, the final color is computed using four pixels that overlap the quantized position addressed on the forward frame KFF. The same applies to the backward frame KFB. A total of 8 pixels are used with weighting factors wi con i=1 . . . 8. If only one motion vector is valid .... Thus, a set of v(xc,yc,z.sc) motion vectors is selected), wherein the computing the pixel data comprises computing the pixel data using the selected subset of one or more motion vectors (Paragraph [0140], if only one motion vector is valid, this is used to compute the color of the pixel. Only 4 pixels and weighting factors are needed, related to a fully rendered frame KF).
Regarding claim 13, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 12), and Barone further disclose wherein the selecting comprises selecting the subset of one or more motion vectors (FIG. 6; paragraph [0106], v(x0,y0,z.0) is the notation for a generic motion vector that expresses the motion between a vertex v for the (x1,y1,z1) position at the time instant t1, related to the only interpolated frame available IF ...; paragraphs [0139]-[0140], if a pixel (i,j) is not in the background, a check operation is performed to test if the motion vectors are valid or not .. After such operation if both motion vectors for the pixel (i,j) are valid, the final color is computed using four pixels that overlap the quantized position addressed on the forward frame KFF. The same applies to the backward frame KFB. A total of 8 pixels are used with weighting factors wi con i=1 . . . 8. If only one motion vector is valid .... Thus, a set of v(xc,yc,z.sc) motion vectors is selected from first motion vectors or second motion vectors) from the one or more first motion vectors (Paragraph [0106], v(x0,y0,z.0) is the notation for a generic motion vector that expresses the motion between a vertex v for the (x1,y1,z1) position at the time instant t1, related to the only interpolated frame available IF) or the one or more second motion vectors (Paragraph [0106], the motion between a vertex v at the position (x1,y1,z1) at the instant t1 (again related to the interpolated frame IF) and the position (x2,y2,z2) at the time instant t2).
Regarding claim 15, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 11), and Barone further disclose wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational Al operations;
a system (Paragraph [0190], FIG. 18 shows how a double perspective correction module 400 as described herein can be integrated in a HW accelerated 3D engine system supported by a bus 402 and including a host CPU 404) for generating synthetic data (FIG. 3; paragraph [0073], the motion buffer encoder 214 computes the difference between incoming motion vectors. The difference is then coded using a variable length code (VLC) in the form of coded motion buffer information in a bitstream R4 that is sent through a transmitter block 224 on a wireless or wired channel to a remote smart display 230. A video encoder 215 encodes the fully rendered frames KF output at a full rate F2 by the 3D engine 210 and sends a bitstream R3 to a transmitter block 223 that transmits such a bitstream R3 to the smart display 230);
a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
Regarding claim 16, Barone discloses at least one processor (Paragraph [0190], FIG. 18 shows how a double perspective correction module 400 as described herein can be integrated in a HW accelerated 3D engine system supported by a bus 402 and including a host CPU 404) comprising:
one or more circuits to stream, to one or more clients (FIG. 3; paragraph [0073], the motion buffer encoder 214 computes the difference between incoming motion vectors. The difference is then coded using a variable length code (VLC) in the form of coded motion buffer information in a bitstream R4 that is sent through a transmitter block 224 on a wireless or wired channel to a remote smart display 230. A video encoder 215 encodes the fully rendered frames KF output at a full rate F2 by the 3D engine 210 and sends a bitstream R3 to a transmitter block 223 that transmits such a bitstream R3 to the smart display 230), video data (Paragraph [0074], considering a frame comprised of a bi-dimensional matrix of N rows by M columns of RGB colors at a reduced frame rate F2 and using a compression method adapted to compress 24 bits to 6 bits, the bitstream R3 has a rate equal to (N*M*6*F2)) including pixel data corresponding to at least one location of an intermediate frame to one or more first frames and one or more second frames (FIGS. 4, 5 and 6; paragraph [0123], the location (i,j) of the forward motion buffer MBF includes three coordinates (x0,y0,z0)i,j related to the (i,j) pixel in the interpolated frame IF that point at the forward frame KFF), the pixel data computed (Paragraph [0140], if only one motion vector is valid, this is used to compute the color of the pixel. Only 4 pixels and weighting factors are needed, related to a fully rendered frame KF) based at least on:
an inconsistency (Paragraphs [0087]-[0088], in order to generate the interpolated frames IF in succession, a fully rendered forward frame KFF and a backward frame KFB should be available. A backward frame KFB is a reference frame at a later time in display order, while a forward frame KFF is a reference frame at an earlier time in display order ... As its input, the motion compensation stage 237 thus receives a fully rendered frame KF, the frame interpolation index and the motion buffer information MB, and comprises a first buffer (designated a forward motion buffer MBF; paragraph [0123], the location (i,j) of the forward motion buffer MBF includes three coordinates (x0,y0,z0)i,j related to the (i,j) pixel in the interpolated frame IF that point at the forward frame KFF. Similarly the location (i,j) in the backward motion buffer MBB includes (xn+1,yn+1,zn+1)i,j coordinates related to the (i,j) pixel in the interpolated frame IF that point at the backward frame KFB) associated to a forward frame KFF, and a second buffer (designated a backward motion buffer MBB) associated to a backward frame KFB; paragraphs [0138]-[0140], the motion compensation stage 237 performs a general process for computing colors having as its inputs i) two frames KF (forward frame KFF and backward frame KFB) already rendered by the rendering stage 213, to be used as references; ... If a pixel (i,j) is not in the background, a check operation is performed to test if the motion vectors are valid or not .. After such operation if both motion vectors for the pixel (i,j) are valid, the final color is computed using four pixels that overlap the quantized position addressed on the forward frame KFF. The same applies to the backward frame KFB. A total of 8 pixels are used with weighting factors wi con i=1 . . . 8. If only one motion vector is valid ... Thus, an inconsistency between motion vectors is identified if only one motion vector is valid) between one or more first motion vectors (Paragraph [0106], v(x0,y0,z.0) is the notation for a generic motion vector that expresses the motion between a vertex v for the (x1,y1,z1) position at the time instant t1, related to the only interpolated frame available IF) mapping the one or more first frames (Paragraph [0105], the forward frame KFF) to the intermediate frame (Paragraph [0106], interpolated frame IF) and one or more second motion vectors (Paragraph [0106], the motion between a vertex v at the position (x1,y1,z1) at the instant t1 (again related to the interpolated frame IF) and the position (x2,y2,z2) at the time instant t2) mapping the one or more second frames (Paragraph [0106], the backward frame KFB) to the intermediate frame (Paragraph [0106], interpolated frame IF).
However, Barone does not specifically disclose a comparison between at least one motion vector corresponding to the one or more first frames to a reference motion vector corresponding to the one or more first frames.
In additional, Fishwick discloses (Paragraph [0003], FIG. 1 shows a moving object 110 at a certain position in one frame of a video sequence, and, superimposed onto the same figure, the same object 115, at its position in the previous frame in the sequence …; paragraph [0051], FIG. 8 shows three examples, labeled A), B) and C)) a comparison between at least one motion vector (Paragraph [0059], example B) shows the ended vectors 850 and 860) corresponding to the one or more first frames (Paragraph [0051], , in FIG. 8, an interpolated frame 801 has an image space which is divided into a plurality of regions … The scene to which motion estimation is being applied includes objects that are generally moving from the bottom to the top of the frame. In particular, lower block 820 represents the position of an object, or part of an object, 825 in a first frame, and upper block 830 represents the position of the same object portion 835 in a second frame. The interpolated frame 801 is to be an interpolation between the first frame and the second frame) to a reference motion vector corresponding to the one or more first frames (Paragraph [0059], one set of properties associated with the remote double ended vectors 850 and 860 that may be analysed are those of the vectors themselves. Specifically, the magnitude and/or the direction of the remote vectors can be determined and compared with the magnitude and/or direction of the candidate vector 840 to ascertain whether the vectors are the same or different … a global motion pan caused by something like a camera pan. Although a small amount of variation can be expected because things do not always move by exactly the same amount (e.g. the camera pan may actually be a rotation where the exact angle and distance of the vector is dependent on the distance from the camera to the object being tracked), as the difference between the vectors increases it becomes less certain that the objects are moving in the same way).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the pipelined graphic engine for generating image frames for display taught by Barone incorporate the teachings of Fishwick, and applying the motion estimation taught by Fishwick to provide the double ended vectors for motion estimation between two frames; detect a conflict between motion vectors; perform the comparison between motion vectors and global motion to generate pixel data corresponding to the intermediate frame. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Barone according to the relied-upon teachings of Fishwick to obtain the invention as specified in claim.
Regarding claim 17, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 16), and Barone disclose wherein the one or more circuits are to select, based at least on the comparison, a subset of one or more motion vectors from the one or more first motion vectors and the one or more second motion vectors (FIG. 6; paragraph [0106], v(x0,y0,z.0) is the notation for a generic motion vector that expresses the motion between a vertex v for the (x1,y1,z1) position at the time instant t1, related to the only interpolated frame available IF ...; paragraph [0109], by considering a vertex v of a scene triangle having (xc,yc,z.sc) as a set of three-dimensional coordinates specified during the execution time of the application ...; paragraphs [0139]-[0140], if a pixel (i,j) is not in the background, a check operation is performed to test if the motion vectors are valid or not .. After such operation if both motion vectors for the pixel (i,j) are valid, the final color is computed using four pixels that overlap the quantized position addressed on the forward frame KFF. The same applies to the backward frame KFB. A total of 8 pixels are used with weighting factors wi con i=1 . . . 8. If only one motion vector is valid .... Thus, a set of v(xc,yc,z.sc) motion vectors is selected), wherein the pixel data is computed using the selected subset of one or more motion vectors (Paragraph [0140], if only one motion vector is valid, this is used to compute the color of the pixel. Only 4 pixels and weighting factors are needed, related to a fully rendered frame KF).
Regarding claim 18, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 17), and Barone further disclose wherein the motion vectors selected for the subset (FIG. 6; paragraph [0106], v(x0,y0,z.0) is the notation for a generic motion vector that expresses the motion between a vertex v for the (x1,y1,z1) position at the time instant t1, related to the only interpolated frame available IF ...; paragraphs [0139]-[0140], if a pixel (i,j) is not in the background, a check operation is performed to test if the motion vectors are valid or not .. After such operation if both motion vectors for the pixel (i,j) are valid, the final color is computed using four pixels that overlap the quantized position addressed on the forward frame KFF. The same applies to the backward frame KFB. A total of 8 pixels are used with weighting factors wi con i=1 . . . 8. If only one motion vector is valid .... Thus, a set of v(xc,yc,z.sc) motion vectors is selected from first motion vectors or second motion vectors) are from the one or more first motion vectors (Paragraph [0106], v(x0,y0,z.0) is the notation for a generic motion vector that expresses the motion between a vertex v for the (x1,y1,z1) position at the time instant t1, related to the only interpolated frame available IF) or the one or more second motion vectors (Paragraph [0106], the motion between a vertex v at the position (x1,y1,z1) at the instant t1 (again related to the interpolated frame IF) and the position (x2,y2,z2) at the time instant t2).
Regarding claim 20, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 16), and Barone further disclose wherein the at least one processor is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational Al operations;
a system (Paragraph [0190], FIG. 18 shows how a double perspective correction module 400 as described herein can be integrated in a HW accelerated 3D engine system supported by a bus 402 and including a host CPU 404) for generating synthetic data (FIG. 3; paragraph [0073], the motion buffer encoder 214 computes the difference between incoming motion vectors. The difference is then coded using a variable length code (VLC) in the form of coded motion buffer information in a bitstream R4 that is sent through a transmitter block 224 on a wireless or wired channel to a remote smart display 230. A video encoder 215 encodes the fully rendered frames KF output at a full rate F2 by the 3D engine 210 and sends a bitstream R3 to a transmitter block 223 that transmits such a bitstream R3 to the smart display 230);
a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
Claims 4, 14 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Barone (U.S. Patent Application Publication 2007/0285430 A1) in view of Fishwick (U.S. Patent Application Publication 2016/0021386 A1) in view of Guo et al (U.S. Patent Application Publication 2015/0022677 A1).
Regarding claim 4, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 1).
However, Barone does not specifically disclose further comprising computing the reference motion vector based at least on a statistical combination of a plurality of motion vectors.
In additional, Guo discloses further comprising computing the reference motion vector based at least on a statistical combination of a plurality of motion vectors (FIG. 2; paragraph [0041], the global motion vector mv(t) can be computed by solving the MSE cost function: … Equation 1) between the first reference frame and the second reference frame (Paragraph [0040], the method 200 may begin by obtaining one or more consecutive video frames from a camera at block 204; paragraph [0041], An optical flow method can help track corners from frame t to t+1 and obtain N corresponding corner pairs: (p1, q1), (p2, q2), . . . , (pN, qN)).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the pipelined graphic engine for generating image frames for display taught by Barone in view of Fishwick incorporate the teachings of Guo to compute the global motion vector by combining of motion vectors between the first frame and the second frame. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Barone in view of Fishwick according to the relied-upon teachings of Guo to obtain the invention as specified in claim.
Regarding claim 14, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 11).
However, Barone does not specifically disclose further comprising computing the reference motion vector based at least on a statistical combination of a plurality of motion vectors.
In additional, Guo discloses further comprising computing the reference motion vector based at least on a statistical combination (FIG. 2; paragraph [0041], the global motion vector mv(t) can be computed by solving the MSE cost function: … Equation 1) of a plurality of motion vectors (Paragraph [0040], the method 200 may begin by obtaining one or more consecutive video frames from a camera at block 204; paragraph [0041], An optical flow method can help track corners from frame t to t+1 and obtain N corresponding corner pairs: (p1, q1), (p2, q2), . . . , (pN, qN)).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the pipelined graphic engine for generating image frames for display taught by Barone in view of Fishwick incorporate the teachings of Guo to compute the global motion vector by combining of motion vectors between the first frame and the second frame. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Barone in view of Fishwick according to the relied-upon teachings of Guo to obtain the invention as specified in claim.
Regarding claim 19, the combination of Barone in view of Fishwick discloses everything claimed as applied above (see claim 16).
However, Barone does not specifically disclose wherein the reference motion vector corresponds to a statistical combination of a plurality of motion vectors.
In additional, Guo discloses wherein the reference motion vector corresponds to a statistical combination (FIG. 2; paragraph [0041], the global motion vector mv(t) can be computed by solving the MSE cost function: … Equation 1) of a plurality of motion vectors (Paragraph [0040], the method 200 may begin by obtaining one or more consecutive video frames from a camera at block 204; paragraph [0041], An optical flow method can help track corners from frame t to t+1 and obtain N corresponding corner pairs: (p1, q1), (p2, q2), . . . , (pN, qN)).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the pipelined graphic engine for generating image frames for display taught by Barone in view of Fishwick incorporate the teachings of Guo to compute the global motion vector by combining of motion vectors between the first frame and the second frame. Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Barone in view of Fishwick according to the relied-upon teachings of Guo to obtain the invention as specified in claim.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Xilin Guo whose telephone number is (571)272-5786. The examiner can normally be reached Monday - Friday 9:00 AM-5:30 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XILIN GUO/ Primary Examiner, Art Unit 2616
