DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claim(s) 1-20 is/are rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1-7, 11-16, 20 of U.S. Patent No. 12,231,609 (hereinafter ‘609). Although the claims at issue are not identical, they are not patentably distinct from each other because they are viewed as doing the same thing: applying a 3D effect and generating a 3D message based on the 3D effect.
In regards to claim(s) 1, 11, 20, the tables below map correspondence between the limitations of the independent claims of the instant application and limitations of independent claims 1, 11, 20 of ‘609.
Claim 1 of Instant Application
Claim 1 of ‘609
A method, comprising:
A method, comprising:
applying, to image data and depth data, a 3D effect based at least in part on an augmented reality content generator, the applying the 3D effect comprising:
applying, to image data and depth data, a 3D effect based at least in part on an augmented reality content generator, the applying the 3D effect comprising:
generating a depth map using at least the depth data,
Note 1: corresponds to claims 2, 12 of Instant Application
generating a segmentation mask based at least on the image data, and
Note 2: corresponds to claims 3, 13 of Instant Application
converting a single channel floating point texture into a raw depth map;
Note 3: corresponds to bolded portion on the right side
performing background inpainting and blurring of the image data using at least a segmentation mask to generate background inpainted image data, the performing the background inpainting comprising performing a diffusion based inpainting technique that fills in a missing region by propagating image content from a boundary between the missing region and a background region to an interior of the missing region, wherein the background region comprises a particular region of the image data without a foreground subject and the missing region includes the foreground subject; and
performing background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data, the performing the background inpainting comprising performing a diffusion based inpainting technique that fills in a missing region by propagating image content from a boundary between the missing region and a background region to an interior of the missing region, wherein the background region comprises a particular region of the image data without a foreground subject and the missing region includes the foreground subject, generating the depth map comprises converting a single channel floating point texture into a raw depth map, and portions of the single channel floating point texture are sent into multiple lower precision channels; and
Note 4: underlined portion corresponds to claims 4, 14 of Instant Application
generating a 3D message based at least in part of the applied 3D effect.
generating a 3D message based at least in part on the applied 3D effect.
In regards to system claim 11 and medium claim 20, claim(s) 11, 20 recite(s) limitations that is/are similar in scope to the limitations recited in claim 1. Therefore, claim(s) 11, 20 is/are subject to rejections under the same rationale as applied hereinabove for claim 1. Furthermore, correspondence between system claim 11 and medium claim 20 of the instant application can also be made to system claim 11 and medium claim 20 of ‘609 respectively, similar to the correspondence made above for method claim 1 of the instant application to method claim 1 of ‘609.
Further, correspondence of the following dependent claims of the instant application can be made as follows:
claim(s) 2, 12 to claim(s) 1, 11 of ‘609 (See Note 1 above);
claim(s) 3, 13 to claim(s) 1, 11 of ‘609 (See Note 2 above);
claim(s) 4, 14 to claim(s) 1, 11 of ‘609 (See Note 4 above);
claim(s) 5, 15 to claim(s) 2, 12 of ‘609;
claim(s) 6, 16 to claim(s) 3, 13 of ‘609;
claim(s) 7, 17 to claim(s) 4, 14 of ‘609;
claim(s) 8, 18 to claim(s) 5, 15 of ‘609;
claim(s) 9, 19 to claim(s) 6, 16 of ‘609; and
claim(s) 10 to claim(s) 7 of ‘609.
Claim(s) 1-20 is/are rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1-7, 11-16, 20 of U.S. Patent No. 11,825,065 (hereinafter ‘065). Although the claims at issue are not identical, they are not patentably distinct from each other because they are viewed as doing the same thing: applying a 3D effect and generating a 3D message based on the 3D effect.
In regards to claim(s) 1, 11, 20, the tables below map correspondence between the limitations of the independent claims of the instant application and limitations of independent claims 1, 11, 20 of ‘065.
Claim 1 of Instant Application
Claim 1 of ‘065
A method, comprising:
A method, comprising:
receiving, at a client device, a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator including a 3D effect;
applying, to image data and depth data, a 3D effect based at least in part on an augmented reality content generator, the applying the 3D effect comprising:
applying, to image data and depth data, the 3D effect based at least in part on the augmented reality content generator, the applying the 3D effect comprising:
generating a depth map using at least the depth data,
Note 5: corresponds to claims 2, 12 of Instant Application
generating a segmentation mask based at least on the image data, and
Note 6: corresponds to claims 3, 13 of Instant Application
converting a single channel floating point texture into a raw depth map;
Note 7: corresponds to bolded portion on the right side
performing background inpainting and blurring of the image data using at least a segmentation mask to generate background inpainted image data, the performing the background inpainting comprising performing a diffusion based inpainting technique that fills in a missing region by propagating image content from a boundary between the missing region and a background region to an interior of the missing region, wherein the background region comprises a particular region of the image data without a foreground subject and the missing region includes the foreground subject; and
performing background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data, the performing the background inpainting comprising performing a diffusion based inpainting technique that fills in a missing region by propagating image content from a boundary between the missing region and a background region to an interior of the missing region, wherein the background region comprises a particular region of the image data without a foreground subject and the missing region includes the foreground subject, generating the depth map comprises converting a single channel floating point texture into a raw depth map, and portions of the single channel floating point texture are sent into multiple lower precision channels; and
Note 8: underlined portion corresponds to claims 4, 14 of Instant Application
generating a 3D message based at least in part of the applied 3D effect.
generating a 3D message based at least in part on the applied 3D effect.
In regards to system claim 11 and medium claim 20, claim(s) 11, 20 recite(s) limitations that is/are similar in scope to the limitations recited in claim 1. Therefore, claim(s) 11, 20 is/are subject to rejections under the same rationale as applied hereinabove for claim 1. Furthermore, correspondence between system claim 11 and medium claim 20 of the instant application can also be made to system claim 11 and medium claim 20 of ‘065 respectively, similar to the correspondence made above for method claim 1 of the instant application to method claim 1 of ‘065.
Further, correspondence of the following dependent claims of the instant application can be made as follows:
claim(s) 2, 12 to claim(s) 1, 11 of ‘065 (See Note 5 above);
claim(s) 3, 13 to claim(s) 1, 11 of ‘065 (See Note 6 above);
claim(s) 4, 14 to claim(s) 1, 11 of ‘065 (See Note 8 above);
claim(s) 5, 15 to claim(s) 2, 12 of ‘065;
claim(s) 6, 16 to claim(s) 3, 13 of ‘065;
claim(s) 7, 17 to claim(s) 4, 14 of ‘065;
claim(s) 8, 18 to claim(s) 5, 15 of ‘065;
claim(s) 9, 19 to claim(s) 6, 16 of ‘065; and
claim(s) 10 to claim(s) 7 of ‘065.
Claim(s) 1-20 is/are rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1-6, 10-14, 18 of U.S. Patent No. 11,457,196 (hereinafter ‘196). Although the claims at issue are not identical, they are not patentably distinct from each other because they are viewed as doing the same thing: applying a 3D effect and generating a 3D message based on the 3D effect.
In regards to claim(s) 1, 11, 20, the tables below map correspondence between the limitations of the independent claims of the instant application and limitations of independent claims 1, 10, 18 of ‘196.
Claim 1 of Instant Application
Claim 1 of ‘196
A method, comprising:
A method, comprising:
selecting a set of augmented reality content generators from a plurality of available augmented reality content generator based on metadata associated with each respective augmented reality content generator, the metadata including information indicating a corresponding augmented reality content generator includes at least a 3D effect, the set of augmented reality content generators including at least one augmented reality content generator without a 3D effect and at least one augmented reality content generator with a 3D effect;
receiving, at a client device, a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator including a 3D effect;
capturing image data and depth data using at least one camera of the client device; and
Note 9: corresponds to the first limitation of claim 5, 15 of the instant application
applying, to image data and depth data, a 3D effect based at least in part on an augmented reality content generator, the applying the 3D effect comprising:
applying, to the image data and the depth data, the 3D effect based at least in part on the augmented reality content generator, the applying the 3D effect comprising:
generating a depth map using at least the depth data,
Note 10: corresponds to claims 2, 12 of Instant Application
generating a segmentation mask based at least on the image data, and
Note 11: corresponds to claims 3, 13 of Instant Application
converting a single channel floating point texture into a raw depth map;
Note 12: corresponds to bolded portion on the right side
performing background inpainting and blurring of the image data using at least a segmentation mask to generate background inpainted image data, the performing the background inpainting comprising performing a diffusion based inpainting technique that fills in a missing region by propagating image content from a boundary between the missing region and a background region to an interior of the missing region, wherein the background region comprises a particular region of the image data without a foreground subject and the missing region includes the foreground subject; and
performing background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data, the performing the background inpainting comprising performing a diffusion based inpainting technique that fills in a missing region by propagating image content from a boundary between the missing region and a background region to an interior of the missing region, wherein the background region comprises a particular region of the image data without a foreground subject and the missing region includes the foreground subject, generating the depth map comprises converting a single channel floating point texture into a raw depth map, portions of the single channel floating point texture are sent into multiple lower precision channels, the raw depth map has a lower resolution than the image data;
Note 13: underlined portion corresponds to claims 4, 14 of Instant Application
generating a 3D message based at least in part of the applied 3D effect.
generating a 3D message based at least in part on the applied 3D effect; and
rendering a view of the 3D message based at least in part on the applied 3D effect.
Note 14: corresponds to claims 5, 15 of Instant Application
In regards to system claim 10 and medium claim 18, claim(s) 10, 18 recite(s) limitations that is/are similar in scope to the limitations recited in claim 1. Therefore, claim(s) 10, 18 is/are subject to rejections under the same rationale as applied hereinabove for claim 1. Furthermore, correspondence between system claim 11 and medium claim 20 of the instant application can also be made to system claim 10 and medium claim 18 of ‘196 respectively, similar to the correspondence made above for method claim 1 of the instant application to method claim 1 of ‘196.
Further, correspondence of the following dependent claims of the instant application can be made as follows:
claim(s) 2, 12 to claim(s) 1, 10 of ‘196 (See Note 10 above);
claim(s) 3, 13 to claim(s) 1, 10 of ‘196 (See Note 11 above);
claim(s) 4, 14 to claim(s) 1, 10 of ‘196 (See Note 13 above);
claim(s) 5, 15 to claim(s) 1, 10 of ‘196 (See Note 14 above);
claim(s) 6, 16 to claim(s) 1-2, 10-11 of ‘196 (See Note 9 above);
claim(s) 7, 17 to claim(s) 3, 12 of ‘196;
claim(s) 8, 18 to claim(s) 4, 13 of ‘196;
claim(s) 9, 19 to claim(s) 5, 14 of ‘196; and
claim(s) 10 to claim(s) 6 of ‘196.
Allowable Subject Matter
Claim(s) 1-20 is/are allowed over the prior art. However, claims 1-20 are rejected under Double Patenting above.
The following is a statement of reasons for the indication of allowable subject matter:
Claim(s) 1-20 was/were carefully reviewed and a search has been made. Accordingly, those claim(s) are believed to be distinct from the prior art searched.
Regarding claim(s) 1-20 (and specifically independent claim(s) 1, 11, 20), the prior art search was found to neither anticipate nor suggest a method/system/medium, comprising: applying, to image data and depth data, a 3D effect based at least in part on an augmented reality content generator, the applying the 3D effect comprising: converting a single channel floating point texture into a raw depth map; performing background inpainting and blurring of the image data using at least a segmentation mask to generate background inpainted image data, the performing the background inpainting comprising performing a diffusion based inpainting technique that fills in a missing region by propagating image content from a boundary between the missing region and a background region to an interior of the missing region, wherein the background region comprises a particular region of the image data without a foreground subject and the missing region includes the foreground subject – in combination with other claimed limitations (emphasis added).
It is viewed that any of the previously cited references or any of the prior art searched, in part or in whole, cannot be combined in such a way to render the claimed invention obvious.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Piemonte et al. (US 2019/0318540 A1 – augmented reality messaging with 3D objects).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JED-JUSTIN IMPERIAL whose telephone number is (571)270-5807. The examiner can normally be reached Monday to Friday, 9am - 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
/JED-JUSTIN IMPERIAL/ Examiner, Art Unit 2616
Prosecution Insights