DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application is being examined under the pre-AIA first to invent provisions.
Examiner’s Comments
Liu (US 20100014781 A1) is used in rejecting claim 9 however it may be applicable to claim 1 as well, though not relied upon at this time
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, the multiple SIMD processor implementation per claims 1 and 5 must be shown or the feature(s) canceled from the claim(s). No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 12058306. Although the claims at issue are not identical, they are not patentably distinct from each other because the application claim 1 claims a broader version of the same system claimed by the patent claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-4,6-8, rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Zhang (US 20110096832 A1), and further in view of Chen (US 5535288 A).
As per claim 1, Zhang discloses a method comprising:
receiving an ordered set of two-dimensional images representing a moving physical object from a first perspective in a space (para 70, the object and 2d video input );
analyzing the ordered set of two-dimensional images (the 2d video) to determine a motion vector of the moving physical object in the space (any of the motion vectors per para 70) , a depth of the moving physical object (depth estimation per para 70), and the first perspective (para 70, part of : the process of estimating object movement through adjacent video frames, using the perspective geometry per para 66, and the different views per para 91); and
automatically obtaining information extrinsic to the ordered set of two dimensional images based on at least an identification of the moving physical object (para 77 the extrinsic matrices as used for the parameterized model/automatically obtained);
determining a state of hidden surface of the moving physical object based on at least the ordered set of two dimensional images and the extrinsic information (the state of any of the parameters indicating pixel occlusion per the synthesis described in para 83 and 84),
generating a synthetic view of the moving physical object (the objects per para 74) from a different/second perspective (via the 3d view), using a dsp (para 144) dependent on at least the ordered set of two dimensional images (the images as used for the depth map), the motion vector (the motion vector can be used to estimate depth which can be used in the 3d view per para 68);
and the determined state of the hidden surface of the moving physical object (per the pixel synthesis in para 83 and 84).
However, Zhang does not disclose using at least a single instruction, multiple data processor implementation of the dsp per para 20, teaching that it is applicable to the video processing system.
Chen teaches the very well known standard of SIMD that is used in dsp’s in video motion vector estimation for audio and video decoding (para 20).
It would have been obvious to one skilled in the art at the time of filing to implement a well known and widely used processor architecture as SIMD in the DSP of Zhang for the purpose of implementing the video based processing.
As per claim 2, the method according to claim 1, wherein the synthetic view comprises a second ordered set of two-dimensional images from the different perspective and having the same motion vector (the 3d views can be in the context of h.264 per para 87: 3D input which is represented by two views).
As per claim 3, the method according to claim 2, wherein the synthetic view together with the ordered set of two-dimensional images comprise a stereoscopic video image (the 3d image represented by two views per the claim 2 rejection).
As per claim 4, the method according to claim 1, further comprising generating a three dimensional model of the object from the ordered set of two-dimensional images (the depth maps cited in the above rejections as used for the 2d to 3d conversion per para 76).
As per claim 6, the method according to claim 1, wherein the synthetic view comprises stereoscopic image pairs (para 75 synthesized stereo pairs).
As per claim 7, the method according to claim 1, wherein the synthetic view comprises a second ordered set of two-dimensional images representing the moving object from the different perspective (para 75 synthesized stereo pairs versed on the cited motion vector based processing cited above).
As per claim 8, the method according to claim 1, obtaining information extrinsic to the ordered set of 2d images comprises: looking up a record associated with the moving object to determine a state of a hidden surface in at least one two-dimensional image (looking up the record of the parameters used to perform the processes in para 81,82, regarding the 3d warping and or the orientation processing, where orientation and 3d warping are each determined states of the hidden surfaces (the parts of the 3d representation of the objects that are not currently In view), in combination with the extrinsic parameters in para 66 per the extrinsic matrices per para 77).
The following claims 9,11,12,13,15, 16,17,20 are rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Zhang (US 20110096832 A1), and further in view of Liu (US 20100014781 A1).
As per claim 9, the claim 1 rejection discloses a method comprising:
receiving a representation of a two-dimensional image (para 70, the object and 2d video input );
automatically identifying a moving object in the two dimensional image having a hidden surface (the objects and the motion vectors and the occluded pixel processing/synthesizing per the claim 1 rejection) ;
obtaining information extrinsic to the two dimensional image describing the identified moving object (extrinsic parameters per para 66),
determining a perspective and a depth gradient of the two-dimensional image (the orientation and depth maps per para 41, and the perspective via any of the parameters for the orientation based processing per para 78);
predicting a characteristic of a hidden surface of at least one object in the two-dimensional image (the estimated depth maps and or additional views per para 54) based on the obtained extrinsic informaton; and
visually representing the moving object as including a predicted characteristic of the hidden surface from a second perspective different from the first perspective (the synthesized 3d pairs of video images per para 75, based on all of the estimations/predictions made, further noting the pixel synthesis, para 83, 84).
However, Zhang does not disclose using neural network based pattern recognition to perform the processes cited above, including the moving object and hidden surface based processing cited above.
Liu teaches that video processors can use neural network pattern recognition and additional extrinsic information for object identification (para 56). It would have been obvious to one skilled in the art at the time of filing that that the functions of Zhang could be implemented via a neural network for the purpose of automatic distinguishing to identify the objects as disclosed by Zhang.
As per claim 11, the method according to claim 9, further comprising transforming the perspective of the two-dimensional image to a different perspective prior to visually representing the moving object as the stereoscopic images (the 3d warping process per para 76, or the orientation based processing per para 78).
As per claim 12, the method according to claim 11, wherein the transforming/visually representing comprises converting the two-dimensional image to a three dimensional image (the generated 3d sequence per para 91 and any additional processing required to playback the video).
As per claim 13, the method according to claim 11, wherein two dimensional image comprises a video image, and the transformation/visually representing of the moving object occurs in real time at a rate of the video image (para. 28, the system can be part of a codec used in video communications which requires said processing be performed at a real time rate, for the purpose of allowing video communication).
As per claim 15, the system of the claim 1 rejection requires a non-transitory computer readable medium, comprising:
instructions for automatically (the system can be used as part of communications process where all of the cited functions must be in realtime in order to perform the communications, 29,31) analyzing a set of images to determine at least one perspective view of a moving object in space from a perspective (any of the parameters or inputs used to make the motion vector based on the 2d input per the claim 1 rejection, in addition to the multiple views and view synthesis disclosed in para 91 );
instructions for automatically (same reasoning as cited above) determining a characteristic of the moving object in the set of images (the depth, location, or orientation determinations in addition to the motion vector, which indicates a moving object, per the claim 1 rejection);
instructions for automatically identifying the moving object by pattern recognition and obtaining information extrinsic to the set of images by describing the moving object from a database (per the object classification taught by Liu per the claim 9 rejection, and also per the processing of the claim 1 rejection as performed by Zhang);
instructions for predicting a state of a hidden surface of the moving object in at least one image of the set of images based on at least the characteristic and the information extrinsic to the set of images describing the moving object (the synthesized 3d view comprises a predicted state of the hidden or depth surface of an object as part of the 3d video noting para 34 (2d to 3d conversion); and
instructions for generating an output image representing the moving object from a different perspective comprising a view of at least a portion of the hidden surface based on the predicted state of the hidden surface (the 3d warping and view synthesis per para 76 and 77 provides a hidden surface and also the conversion from 2d to 3d via adding depth per para 34 and the claim 1 rejection; additionally per para 91 of Zhang, per the synthesized views).
As per claim 16, the non-transitory computer readable medium according to claim 15, wherein the output image comprises a stereoscopic image (the 3d video cited above).
As per claim 17, the non-transitory computer readable medium according to claim 16, further comprising instructions for generating a three-dimensional model of the object, wherein the output image is generated dependent on the three-dimensional model (the parameters supporting the synthesizing based processing cited in the claim 1 rejection in order to synthesize a 3d views from a 2d view via a depth map and motion vectors) (additionally, the detection as taught by Liu in order to detect an object to present a 3d object).
As per claim 20, the non-transitory computer readable medium according to claim 15, further comprising instructions for determining a motion vector of the object (per the claim 1 rejection), wherein the output image comprises a synthetic view of the object dependent on the determined motion vector (the synthesized 3d image based on the 2d image and the motion vector per the synthesized view of Zhang para 91, ) (alternatively, the synthetic 3d image as taught by Liu).
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5,14 are rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Zhang (US 20110096832 A1), and further in view of Chen (US 5535288 A), as applied to claim 1 above.
As per claim 5, Zhang discloses the method according to claim 1, wherein the synthetic view of the object from a different perspective is generated via a volumetric data transform (part of the 2d-3d video conversion cited above), but does not specify the use of a single-instruction multiple-data (SIMD) processor which performs the volumetric data transform.
The examiner takes official notice it is well known in the art to implement well known processor architectures to perform the cited 2d-3d video conversion, including the volumetric transforms, for the purpose of conforming to well known processing standards and architectures (further noting the teachings of Chen).
As per claim 14, the method according to claim 11, further comprising transforming information of a series of the two-dimensional images with a single-instruction, multiple-data (SIMD) processor (per the claim 5 rejection).
Claims 10,18,19 rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Zhang (US 20110096832 A1) and further in view of Liu (US 20100014781 A1) as applied to claim 9 above and further in view of Algreatly (US 20100080485 A1).
As per claim 10, Zhang discloses the method according to claim 9, but does not specify wherein the determining the perspective and depth gradient comprises extracting at least one of a vanishing line and a vanishing point.
Algreatly teaches an improved interface for 3d object manipulation by a user including modifying a perspective/orientation and a position which includes depth. Algreatly teaches that vanishing points are extracted and used in processing the perspective and depth/position per para 74,75,78. It would have been obvious to one skilled in the art at the time of filing to extract vanishing points for the purpose of allowing manipulation of the perspective and depth gradient of Zhang for the advantage of an improved interface to manipulate 3d objects.
As per claim 18, the non-transitory computer readable medium according to claim 15, further comprising instructions for extracting at least one depth gradient in the at least one image based on at least one vanishing line (para 70,71, and per the claim 10 rejection) and at least one vanishing point (para 70,71 and per the claim 10 rejection).
As per claim 19, the prior art cited above discloses the non-transitory computer readable medium according to claim 18, however Zhang and Algreatly do not specify: wherein the automatically determining a characteristic of the object comprises performing a lookup of the object in a database.
The examiner takes official notice it is well known in the art to use lookup tables and prestored function results and objects for the purpose of improved processing architectures.
Response to Arguments
The submitted arguments have been considered but are moot in view of the new grounds of rejection.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDER KRZYSTAN whose telephone number is 571-272-7498, and whose email address is alexander.krzystan@uspto.gov
The examiner can usually be reached on m-f 7:30-4:00 est.
If attempts to reach the examiner by telephone or email are unsuccessful, the examiner’s supervisor, Fan Tsang can be reached on (571) 272-7547.
The fax phone numbers for the organization where this application or proceeding is assigned are 571-273-8300 for regular communications and 571-273-8300 for After Final communications.
/ALEXANDER KRZYSTAN/Primary Examiner, Art Unit 2653
Examiner Alexander Krzystan
July 28, 2026
Prosecution Insights