DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Examiner has carefully considered Applicant’s Remarks dated November 24, 2025.
As for Applicant’s arguments regarding various limitations of independent claims 1, 17, and 18 not being taught by Lindsay (Remarks, pages 3-6); a secondary reference is being used in this Office Action.
Accordingly, independent claims 1, 17, and 18 remain rejected. The dependent claims remain rejected as well.
Since the scope of the rejections is being altered without the claims having been amended, this Office Action is non-final.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement.
Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b).
Claims 1-18 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over the claims of Patent No US 12,154,236 B1, in view of Oser (US 2020/0175275 A1). Although the conflicting claims are not identical, they are not patentably distinct from each other because the claims in the instant application and the publication are substantially identical in structural and functional characteristics. For example:
Instant Application
Patent No US 12,154,236 B1
Explanation
1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:
receive, from a user, a first input corresponding to a request for assistance with a drawing; in accordance with receiving the first input, select a physical surface in a physical environment; display, over the selected physical surface, a graphical overlay representing the drawing, wherein displaying the graphical overlay over the selected physical surface includes anchoring the graphical overlay to the selected physical surface;
while displaying the graphical overlay over the selected physical surface, receive, from the user, a second input corresponding to a request to save a layout corresponding to the drawing;
in accordance with receiving the second input, generate the layout corresponding to the drawing, wherein the layout indicates an anchoring of the graphical overlay to the selected physical surface;
and transmit the layout to an external electronic device.
1. An electronic device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving, from a user, an input corresponding to a request for assistance with a drawing; in accordance with receiving the input, selecting a physical surface in a physical environment; displaying, over the selected physical surface, a graphical overlay representing the drawing, wherein displaying the graphical overlay over the selected physical surface includes anchoring the graphical overlay to the selected physical surface;
wherein displaying the graphical overlay over the selected physical surface includes anchoring the graphical overlay to the selected physical surface
Paragraph 59 of Oser teaches a user input for saving an item.
A drawing on one electronic device may obviously be transmitted to a second electronic device – such as a printer, for example.
In regards to instant claims 2-18, all limitations of these claims are contained in the claims of the ‘236 patent, in view of Oser, as follows:
Instant Application
Patent No US 12,154,236 B1
2
3
3
2
4
1
5
1
6
20
7
20
8
14
9
14
10
2
11
3
12
5
13
6
14
7
15
8
16
12
17
1
18
25
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4 and 10-18 are rejected under 35 U.S.C. 103 as being unpatentable over Lindsay (US 2017/0270715 A1) in view of Oser (US 2020/0175275 A1).
Instant Claim 1: A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors (“The controller 908 (fig 9) includes one or more logic devices and one or more computer memory devices storing instructions executable by the logic device(s) to enact functionalities of the head-mounted display device 900.” (Lindsay, paragraph 41) The computer memory device of Lindsay corresponds to the computer-readable storage medium of the claim. The instructions of Lindsay correspond to the instructions and programs of the claim.)
of an electronic device, cause the electronic device to: (“FIG. 1 shows an example use scenario for an example head-mounted display device,” (Lindsay, paragraph 6) The head-mounted display device of Lindsay corresponds to the electronic device of the claim.)
receive, from a user, a first input corresponding to a request for assistance with a drawing; (“One example provides, on a display device comprising a camera and a display, a method comprising acquiring, via the camera, image data imaging an environment, receiving a user input requesting display of a three-dimensional virtual object,” (Lindsay, abstract) The user’s input in Lindsay corresponds to the request of the claim. The three-dimensional virtual object corresponds to the drawing of the claim.)
in accordance with receiving the first input, select a physical surface in a physical environment; (When the camera of Lindsay images the environment, a physical surface in a physical environment is selected.)
display, over the selected physical surface, a graphical overlay representing the drawing, wherein displaying the graphical overlay over the selected physical surface includes anchoring the graphical overlay to the selected physical surface; (“As the user 102 (fig 4) moves the three-dimensional virtual object 400, the three-dimensional virtual object 400 may be displayed such that it appears to “snap to” a surface within the environment 100.” (Lindsay, paragraph 28))
and transmit the layout to an external electronic device. (“In some embodiments, the communication subsystem 1026 (fig 10) may allow the augmented reality display system 1000 to send and/or receive data to and/or from other devices via a network such as the Internet.” (Lindsay, paragraph 54) It would be obvious for the user of Lindsay to be provided the ability to print his design, for instance.)
Lindsay does not explicitly teach the following limitations of this claim:
while displaying the graphical overlay over the selected physical surface, receive, from the user, a second input corresponding to a request to save a layout corresponding to the drawing; in accordance with receiving the second input, generate the layout corresponding to the drawing, wherein the layout indicates an anchoring of the graphical overlay to the selected physical surface;
In a similar field of endeavor, however, Oser teaches a user input for saving an item in a simulated reality environment.
while displaying the graphical overlay over the selected physical surface, receive, from the user, a second input corresponding to a request to save a layout corresponding to the drawing; (“The storage subsystem 1024 (fig 10) may include removable and/or built-in devices. The storage subsystem 1024 may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.). semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), among others. The storage subsystem 1024 may include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices.” (Lindsay, paragraph 50)
“As another example, a second selection input type saves an item to an SR clipboard or environment for viewing at a later time.” (Oser, paragraph 59) The user of Lindsay would obviously like to save in memory the three-dimensional virtual object design he has created, similar to Oser.)
in accordance with receiving the second input, generate the layout corresponding to the drawing, wherein the layout indicates an anchoring of the graphical overlay to the selected physical surface; (“As the user 102 (fig 4) moves the three-dimensional virtual object 400, the three-dimensional virtual object 400 may be displayed such that it appears to “snap to” a surface within the environment 100.” (Lindsay, paragraph 28) When the user saves his design, the entire picture would be saved.)
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the head-mounted display device as taught by Lindsay, wherein a user may place a three-dimensional virtual object on various surfaces; with the simulated environment as taught by Oser, wherein a user may indicate a save input. Such a combination incorporates a known feature (Oser) into a known device in order to yield the predictable result of enabling the user to save his designs in memory for later recall.
Instant Claim 2: The non-transitory computer-readable storage medium of claim 1, wherein the second input includes a speech input corresponding to the request to save the layout corresponding to the drawing. (“A user may interact with a three-dimensional virtual object via any suitable user input mode, including but not limited to speech, gesture, touch, and eye gaze.” (Lindsay, paragraph 27))
Instant Claim 3: The non-transitory computer-readable storage medium of claim 1, wherein the second input corresponds to a selection of an icon displayed concurrently with the graphical overlay. (“FIG. 4 shows an example of a user 102 interacting with a three-dimensional virtual object 400 using a hand gesture 402.” (Lindsay, paragraph 27)
“As another example, a second selection input type saves an item to an SR clipboard or environment for viewing at a later time.” (Oser, paragraph 59))
Instant Claim 4: The non-transitory computer-readable storage medium of claim 1, wherein transmitting the layout to the external electronic device includes transmitting the layout using at least one of: an email message; a text message; and Bluetooth communication between the electronic device and the external electronic device. (“As non-limiting examples, the communication subsystem 1026 (fig 10) may be configured for communication via a wireless telephone network, or a wired or wireless local- or wide-area network. In some embodiments, the communication subsystem 1026 may allow the augmented reality display system 1000 to send and/or receive data to and/or from other devices via a network such as the Internet.” (Lindsay, paragraph 54) Email messages, text messages, and Bluetooth are included in the communication protocols of Lindsay.)
Instant Claim 10: The non-transitory computer-readable storage medium of claim 1, wherein the first input corresponds to a selection of an image corresponding to the drawing or to a selection of a 3-dimensional model corresponding to the drawing. (“FIG. 4 shows an example of a user 102 interacting with a three-dimensional virtual object 400 using a hand gesture 402.” (Lindsay, paragraph 27))
Instant Claim 11: The non-transitory computer-readable storage medium of claim 1, wherein the first input includes a second speech input specifying the drawing. (“A user may interact with a three-dimensional virtual object via any suitable user input mode, including but not limited to speech, gesture, touch, and eye gaze.” (Lindsay, paragraph 27))
Instant Claim 12: The non-transitory computer-readable storage medium of claim 1, wherein selecting the physical surface includes: receiving, from the user, a third input corresponding to a selection of the physical surface, wherein displaying the graphical overlay over the selected physical surface is performed in accordance with receiving the third input. (“FIG. 4 shows an example of a user 102 interacting with a three-dimensional virtual object 400 using a hand gesture 402.” (Lindsay, paragraph 27))
Instant Claim 13: The non-transitory computer-readable storage medium of claim 12, wherein the third input includes a user gaze input, wherein the user gaze input indicates that a user gaze is directed at the physical surface. (“A user may interact with a three-dimensional virtual object via any suitable user input mode, including but not limited to speech, gesture, touch, and eye gaze.” (Lindsay, paragraph 27))
Instant Claim 14: The non-transitory computer-readable storage medium of claim 12, wherein the third input includes a third speech input including a reference to the physical surface, and wherein selecting the physical surface further includes: processing the third speech input using context information associated with the third speech input to determine that the reference refers to the physical surface. (“A user may interact with a three-dimensional virtual object via any suitable user input mode, including but not limited to speech, gesture, touch, and eye gaze.” (Lindsay, paragraph 27))
Instant Claim 15: The non-transitory computer-readable storage medium of claim 12, wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to: in accordance with receiving the first input corresponding to the request for assistance with the drawing, and prior to receiving the third input, display the graphical overlay over the physical environment at a first location, and wherein: the third input corresponds to displacing the graphical overlay from the first location to a second location on the physical surface. (“In the example of FIG. 4. as a user input moves a position of the three-dimensional virtual object 400, the HMD 104 may detect the three-dimensional virtual object 400 being moved to within a threshold distance of a surface of the floor in the environment 100.” (Lindsay, paragraph 29))
Instant Claim 16: The non-transitory computer-readable storage medium of claim 1, wherein anchoring the graphical overlay to the selected physical surface includes: displaying the graphical overlay at a third location on the selected physical surface; detecting movement of the electronic device; and in response to detecting the movement, continuing to display the graphical overlay at the third location. (“In response, the HMD 104 may be configured to display visual feedback 404 (fig 4) indicating that the three-dimensional virtual object 400 can be made to “snap to” the floor surface (e.g. be displayed as in contact with the surface and having movement of the three-dimensional virtual object 400 constrained to be along the surface).” (Lindsay, paragraph 29) Once the three-dimensional virtual object 400 of Lindsay has been put in its place by the user and the application is no longer in the mode of moving the three-dimensional virtual object 400, the head-mounted display device may be moved without the three-dimensional virtual object 400 being moved.)
Instant Claim 17: (Claim 17 is substantially identical to claim 1, and thus, is rejected under similar rationale.)
Instant Claim 18: (Method claim 18 and apparatus claim 1 are related as apparatus and the method of using same, with each claimed element’s function corresponding to the claimed method step. Accordingly, claim 18 is similarly rejected under the same rationale as applied above with respect to apparatus claim 1.)
Claims 5-9 are rejected under 35 U.S.C. 103 as being unpatentable over Lindsay, in view of Oser, and further in view of Migos (US 2016/0357430 A1).
Instant Claim 5: The non-transitory computer-readable storage medium of claim 1, wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to: while displaying the graphical overlay over the selected physical surface: detect a marking produced by the user on the physical surface, the marking corresponding to the drawing. (Lindsay teaches the computer memory device in accordance with claim 1, but does not explicitly disclose a feature wherein a user may mark the three-dimensional virtual object 400. However, in a similar field of endeavor, Migos teaches the use of such a marking feature for a graphical user interface: “In some embodiments, rendering the line in the sketch area in accordance with the first edge of the on-screen ruler includes (610) (fig 6A) tracing the first edge with a predetermined line color in the sketch area while a corresponding on-screen location of a contact of the first touch input moves within a threshold distance from the first edge of the on-screen ruler (e.g., roughly along the first edge of the on-screen ruler). This is illustrated in FIG. 5F, for example.” (Migos, paragraph 290) It would be obvious for such a marking feature in Migos to be available in Lindsay, thus allowing the user in fig 4 of Lindsay to sketch in colors for the chair, for example.)
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the head-mounted display device as taught by Lindsay, wherein a user may place a three-dimensional virtual object on various surfaces; with the graphical user interface as taught by Migos, wherein a user may mark the three-dimensional virtual object. Such a combination incorporates a known feature (Migos) into a known device in order to yield the predictable result of enabling the user to visualize the three-dimensional virtual object in different colors, for example.
Instant Claim 6: The non-transitory computer-readable storage medium of claim 5, wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to: determine that the detected marking corresponds to a completed portion of the drawing; and wherein displaying, over the selected physical surface, the graphical overlay includes: in accordance with a determination that the detected marking corresponds to the completed portion of the drawing, modifying a first display manner of a first portion of the graphical overlay, the first portion corresponding to the completed portion of the drawing. (“The method further includes, in response to detecting a second touch input directed to the first edge of the on-screen ruler, adding fill content in a first portion of the sketch area that corresponds to regions touched by the second touch input on the touch-sensitive surface and that is delineated by the first edge of the on-screen ruler, such that the first portion of the sketch area is confined by the first edge or the line.” (Migos, paragraph 6) Therefore, Migos can tell when the user has completed the marking – and the processor then adds fill content.)
Instant Claim 7: The non-transitory computer-readable storage medium of claim 6, wherein modifying the first display manner includes modifying a color of the first portion. (“In some embodiments, rendering the line in the sketch area in accordance with the first edge of the on-screen ruler includes (610) (fig 6A) tracing the first edge with a predetermined line color in the sketch area while a corresponding on-screen location of a contact of the first touch input moves within a threshold distance from the first edge of the on-screen ruler (e.g., roughly along the first edge of the on-screen ruler). This is illustrated in FIG. 5F, for example.” (Migos, paragraph 290)
Instant Claim 8: The non-transitory computer-readable storage medium of claim 6, wherein displaying the graphical overlay over the selected physical surface includes displaying a first instance of the graphical overlay over the selected physical surface, and wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to: in accordance with modifying the first display manner: update the layout to indicate the modified first display manner, wherein updating the layout causes the external electronic device to: while displaying a second instance of the graphical overlay over the selected physical surface, modify a second display manner of a second portion of the second instance of the graphical overlay according to the modified first display manner, wherein the first portion and the second portion correspond to a same portion of the graphical overlay. (“In some embodiments, the communication subsystem 1026 (fig 10) may allow the augmented reality display system 1000 to send and/or receive data to and/or from other devices via a network such as the Internet.” (Lindsay, paragraph 54) When the external device in Lindsay receives the updated layout, the external device displays the updated layout.)
Instant Claim 9: The non-transitory computer-readable storage medium of claim 8, wherein updating the layout to indicate the modified first display manner is performed in accordance with a determination that the modification to the first display manner is of a predetermined type. (“The method further includes, in response to detecting a second touch input directed to the first edge of the on-screen ruler, adding fill content in a first portion of the sketch area that corresponds to regions touched by the second touch input on the touch-sensitive surface and that is delineated by the first edge of the on-screen ruler, such that the first portion of the sketch area is confined by the first edge or the line.” (Migos, paragraph 6) The adding fill content of Migos may be considered a predetermined type.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Yaron Cohen whose telephone number is (571)270-7995. The examiner can normally be reached Monday - Friday 8:30 AM to 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Temesghen Ghebretinsae can be reached on 571-272-3017. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YARON COHEN/Examiner, Art Unit 2626