DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-21 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-21 of U.S. Patent No. 12,223,627 in view of Lee et al. U.S. Pub. No. 2020/0302664. Claims 2, 9 and 16 of the Instant Application have an additional limitation that is taught by Lee.
Instant Application 19/036,577
U.S. Patent No. 12,223,627
1. A method, comprising:
transmitting, from a client, a request to access image data representing an image;
after transmitting the request, receiving a first portion of the image data, the first portion of the image data representing a first layer,
rendering a smoothed version of the first layer on a display associated with the client by performing a smoothing operation on the first layer, the smoothing operation being based on a parameter specifying an amount of smoothing;
while rendering the smoothed version of the first layer, receiving a second portion of the image data, the second portion of the image data representing a second layer;
and rendering the second layer on the display while displaying the smoothed version of the first layer on the display.
1. A method, comprising:
transmitting, from a client, a request to access image data representing an image;
after transmitting the request, receiving a first portion of the image data, the first portion of the image data representing a first layer including a set of features;
in response to determining that a download time for receiving the image data satisfies a condition:
rendering, as the first layer, a smoothed version of the image on a display associated with the client, the smoothed version of the image including more than one color;
from claim 7
7. The method as in claim 1, wherein rendering the smoothed version of the image on the display includes:
performing a smoothing operation to produce the smoothed version of the image according to a user-specified value of a smoothing parameter.
(claim 1 cont.)
and while rendering the smoothed version of the image on the display associated with the client, receiving a second portion of the image data, the second portion of the image data representing a second layer;
and rendering the second layer on the display while displaying the set of features of the first layer on the display.
2. The method as in claim 1, wherein the smoothing operation is performed on a graphics processing unit of the client.
Lee teaches this limitation. (“Referring to FIG. 1, a multilayered augmented reality image (e.g., multilayered AR image or multilayered AR video) may be created using augmented reality (AR) user device 100, server 200, and various displays 410, 420, and 430... AR user device 100 may be connected with server 200 through network 300...”; Lee, [0046], Fig. 1)
Fig. 1 illustrates that the a multilayered AR image is created using an AR user device (client), a server and multiple displays. The AR user device (client) is connected to the server over network 300.
(“The image processor may be configured to use a Gaussian filter to blur the background image layer... For example, AR user device 100 may make the first image layer blur and overlap the object image layer on the blurred first image layer in order to improve a 3D effect and depth of field in real time.”, Lee, [0013], [0053], Fig. 2)
Fig. 2 illustrates the AR user device (client), that makes the first image layer blur. The image processor, of the AR user device, uses a Gaussian filter to blur the background image layer. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the U.S. Patent by adding the feature of the smoothing operation is performed on a graphics processing unit of the client, in order to improve a 3D effect and depth of field in real time, as taught by Lee ([0053]).
3. The method as in claim 1, wherein the smoothed version of the first layer is rendered over a set of image pixels of the image,
wherein the first layer includes a first set of features, wherein the second portion of the image data further includes a third layer, and wherein the method further comprises: rendering, as the second layer, a second set of features over a first subset of the set of image pixels;
and rendering, as the third layer, a third set of features over a second subset of the set of image pixels.
2. The method as in claim 1, wherein the first layer is rendered over a set of image pixels of the image,
wherein the set of features is a first set of features, wherein the second portion of the image data further includes a third layer, and wherein the method further comprises: rendering, as the second layer, a second set of features over a first subset of the set of image pixels;
and rendering, as the third layer, a third set of features over a second subset of the set of image pixels.
4. The method as in claim 1, wherein the smoothed version of first layer includes a first set of features,
and wherein the method further comprises: rendering, as the second layer, a second set of features over the first set of features.
From claim 2
2. The method as in claim 1, wherein the first layer is rendered over a set of image pixels of the image, wherein the set of features is a first set of features,
3. The method as in claim 1, wherein the first layer is rendered over a set of image pixels of the image, wherein the set of features is a first set of features,
and wherein the method further comprises: rendering, as the second layer, a second set of features over the first layer.
5. The method as in claim 4, wherein the smoothed version of the first layer is rendered over a set of image pixels of the image, and
wherein rendering the second set of features includes:
performing a crossfading operation on a subset of the set of image pixels.
From claim 2
2. The method as in claim 1, wherein the first layer is rendered over a set of image pixels of the image
4. The method as in claim 3, wherein rendering the second set of features includes:
performing a crossfading operation on a subset of the set of image pixels.
6. The method as in claim 5, wherein the crossfading operation includes a convolution operation with a specified blurring kernel.
5. The method as in claim 4, wherein the crossfading operation includes a convolution operation with a specified blurring kernel.
7. The method as in claim 5, wherein the crossfading operation is applied to subblocks of blocks of pixels, the subblocks located at a corner of the blocks.
6. The method as in claim 4, wherein the crossfading operation is applied to subblocks of blocks of pixels, the subblocks located at a corner of the blocks.
8. A computer program product comprising a nontransitive storage medium, the computer program product including code that, when executed by processing circuitry of a computing device, causes the processing circuitry to perform a method, the method comprising:
transmitting, from a client, a request to access image data representing an image;
after transmitting the request, receiving a first portion of the image data, the first portion of the image data representing a first layer,
rendering a smoothed version of the first layer on a display associated with the client by performing a smoothing operation on the first layer, the smoothing operation being based on a parameter specifying an amount of smoothing;
while rendering the smoothed version of the first layer, receiving a second portion of the image data, the second portion of the image data representing a second layer;
and rendering the second layer on the display while displaying the smoothed version of the first layer on the display.
9. A computer program product comprising a nontransitive storage medium, the computer program product including code that, when executed by processing circuitry of a computing device, causes the processing circuitry to perform a method, the method comprising:
transmitting, from a client, a request to access image data representing an image;
after transmitting the request, receiving a first portion of the image data, the first portion of the image data representing a first layer including a set of features,
in response to determining that a download time for receiving the image data satisfies a condition:
rendering, as the first layer, a smoothed version of the image on a display associated with the client, the smoothed version of the image including more than one color;
From claim 15
15. The computer program product as in claim 9, wherein rendering the smoothed version of the image on the display includes: performing a smoothing operation to produce the smoothed version of the image according to a user-specified value of a smoothing parameter.
(claim 9 cont.)
and while rendering the smoothed version of the image on the display associated with the client, receiving a second portion of the image data, the second portion of the image data representing a second layer;
and rendering the second layer on the display while displaying the set of features of the first layer on the display.
9. The computer program product as in claim 8, wherein the smoothing operation is performed on a graphics processing unit of the client.
Lee teaches this limitation. (“Referring to FIG. 1, a multilayered augmented reality image (e.g., multilayered AR image or multilayered AR video) may be created using augmented reality (AR) user device 100, server 200, and various displays 410, 420, and 430... AR user device 100 may be connected with server 200 through network 300...”; Lee, [0046], Fig. 1)
Fig. 1 illustrates that the a multilayered AR image is created using an AR user device (client), a server and multiple displays. The AR user device (client) is connected to the server over network 300.
(“The image processor may be configured to use a Gaussian filter to blur the background image layer... For example, AR user device 100 may make the first image layer blur and overlap the object image layer on the blurred first image layer in order to improve a 3D effect and depth of field in real time.”, Lee, [0013], [0053], Fig. 2)
Fig. 2 illustrates the AR user device (client), that makes the first image layer blur. The image processor, of the AR user device, uses a Gaussian filter to blur the background image layer. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the U.S. Patent by adding the feature of the smoothing operation is performed on a graphics processing unit of the client, in order to improve a 3D effect and depth of field in real time, as taught by Lee ([0053]).
10. The computer program product as in claim 8, wherein the smoothed version of the first layer is rendered over a set of image pixels of the image,
wherein the first layer includes a first set of features, wherein the second portion of the image data further includes a third layer,
and wherein the method further comprises: rendering, as the second layer, a second set of features over a first subset of the set of image pixels;
and rendering, as the third layer, a third set of features over a second subset of the set of image pixels.
10. The computer program product as in claim 9, wherein the first layer is rendered over a set of image pixels of the image,
wherein the set of features is a first set of features, wherein the second portion of the image data further includes a third layer,
and wherein the method further comprises: rendering, as the second layer, a second set of features over a first subset of the set of image pixels;
and rendering, as the third layer, a third set of features over a second subset of the set of image pixels.
11. The computer program product as in claim 8, wherein the smoothed version of first layer includes a first set of features,
and wherein the method further comprises: rendering, as the second layer, a second set of features over the first set of features.
From claim 10
10. The computer program product as in claim 9, wherein the first layer is rendered over a set of image pixels of the image, wherein the set of features is a first set of features,
11. The computer program product as in claim 9, wherein the first layer is rendered over a set of image pixels of the image, wherein the set of features is a first set of features,
and wherein the method further comprises: rendering, as the second layer, a second set of features over the first layer.
12. The computer program product as in claim 11, wherein the smoothed version of the first layer is rendered over a set of image pixels of the image,
and wherein rendering the second set of features includes: performing a crossfading operation on a subset of the set of image pixels.
From claim 10
10. The computer program product as in claim 9, wherein the first layer is rendered over a set of image pixels of the image,
12. The computer program product as in claim 11, wherein rendering the second set of features includes: performing a crossfading operation on a subset of the set of image pixels.
13. The computer program product as in claim 12, wherein the crossfading operation includes a convolution operation with a specified blurring kernel.
13. The computer program product as in claim 12, wherein the crossfading operation includes a convolution operation with a specified blurring kernel.
14. The computer program product as in claim 12, wherein the crossfading operation is applied to subblocks of blocks of pixels, the subblocks located at a corner of the blocks.
14. The computer program product as in claim 12, wherein the crossfading operation is applied to subblocks of blocks of pixels, the subblocks located at a corner of the blocks.
15. An electronic apparatus, the electronic apparatus comprising:
memory;
and a processor coupled to the memory, the processor being configured to:
transmit, from a client, a request to access image data representing an image;
after transmitting the request, receive a first portion of the image data, the first portion of the image data representing a first layer,
render a smoothed version of the first layer on a display associated with the client by performing a smoothing operation on the first layer, the smoothing operation being based on a parameter specifying an amount of smoothing;
while rendering the smoothed version of the first layer, receive a second portion of the image data, the second portion of the image data representing a second layer;
and render the second layer on the display while displaying the smoothed version of the first layer on the display.
16. An electronic apparatus, the electronic apparatus comprising:
memory;
and controlling circuitry coupled to the memory, the controlling circuitry being configured to:
transmit, from a client, a request to access image data representing an image;
after transmitting the request, receive a first portion of the image data, the first portion of the image data representing a first layer, the first layer including a set of features,
and in response to determining that a download time for receiving the image data satisfies a condition:
render, as the first layer, a smoothed version of the image on a display associated with the client, the smoothed version of the image including more than one color;
From claim 22
22. The electronic apparatus as in claim 16, wherein the controlling circuitry configured to render the smoothed version of the image on the display is further configured to: perform a smoothing operation to produce the smoothed version of the image according to a user-specified value of a smoothing parameter.
(claim 16 cont.)
and while rendering the smoothed version of the image on the display associated with the client, receive a second portion of the image data, the second portion of the image data representing a second layer;
and render the second layer on the display while displaying the set of features of the first layer on the display.
16. The electronic apparatus as in claim 15, wherein the smoothing operation is performed on a graphics processing unit of the client.
Lee teaches this limitation. (“Referring to FIG. 1, a multilayered augmented reality image (e.g., multilayered AR image or multilayered AR video) may be created using augmented reality (AR) user device 100, server 200, and various displays 410, 420, and 430... AR user device 100 may be connected with server 200 through network 300...”; Lee, [0046], Fig. 1)
Fig. 1 illustrates that the a multilayered AR image is created using an AR user device (client), a server and multiple displays. The AR user device (client) is connected to the server over network 300.
(“The image processor may be configured to use a Gaussian filter to blur the background image layer... For example, AR user device 100 may make the first image layer blur and overlap the object image layer on the blurred first image layer in order to improve a 3D effect and depth of field in real time.”, Lee, [0013], [0053], Fig. 2)
Fig. 2 illustrates the AR user device (client), that makes the first image layer blur. The image processor, of the AR user device, uses a Gaussian filter to blur the background image layer. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the U.S. Patent by adding the feature of the smoothing operation is performed on a graphics processing unit of the client, in order to improve a 3D effect and depth of field in real time, as taught by Lee ([0053]).
17. The electronic apparatus as in claim 15, wherein the smoothed version of the first layer is rendered over a set of image pixels of the image,
wherein the first layer includes a first set of features, wherein the second portion of the image data further includes a third layer, and wherein the processor is further configured to:
render, as the second layer, a second set of features over a first subset of the set of image pixels; and render, as the third layer, a third set of features over a second subset of the set of image pixels.
17. The electronic apparatus as in claim 16, wherein the first layer is rendered over a set of image pixels of the image,
wherein the set of features is a first set of features, wherein the second portion of the image data further includes a third layer, and wherein the controlling circuitry is further configured to:
render, as the second layer, a second set of features over a first subset of the set of image pixels; and render, as the third layer, a third set of features over a second subset of the set of image pixels.
18. The electronic apparatus as in claim 15, wherein the smoothed version of first layer includes a first set of features,
and wherein the processor is further configured to: render, as the second layer, a second set of features over the first set of features.
From claim 16
after transmitting the request, receive a first portion of the image data, the first portion of the image data representing a first layer, the first layer including a set of features,
18. The electronic apparatus as in claim 16, wherein the first layer is rendered over a set of image pixels of the image, wherein the set of features is a first set of features,
and wherein the controlling circuitry is further configured to: rendering, as the second layer, a second set of features over the first layer.
19. The electronic apparatus as in claim 18, wherein the smoothed version of the first layer is rendered over a set of image pixels of the image,
and wherein the processor configured to render the second set of features is further configured to: perform a crossfading operation on a subset of the set of image pixels.
From claim 17
17. The electronic apparatus as in claim 16, wherein the first layer is rendered over a set of image pixels of the image,
19. The electronic apparatus as in claim 18, wherein the controlling circuitry configured to render the second set of features is further configured to: perform a crossfading operation on a subset of the set of image pixels.
20. The electronic apparatus as in claim 19, wherein the crossfading operation includes a convolution operation with a specified blurring kernel.
20. The electronic apparatus as in claim 19, wherein the crossfading operation includes a convolution operation with a specified blurring kernel.
21. The electronic apparatus as in claim 19, wherein the crossfading operation is applied to subblocks of blocks of pixels, the subblocks located at a corner of the blocks.
21. The electronic apparatus as in claim 19, wherein the crossfading operation is applied to subblocks of blocks of pixels, the subblocks located at a corner of the blocks.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 8, 15; 2, 4, 9, 11, 16, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Frazier et al. U.S. Patent No. 7,127,453 in view of Lee et al. U.S. Pub. No. 2020/0302664, Agrawal et al. U.S. Patent No. 10,217,195 and Jooste U.S. Pub. No. 2013/0050253.
Re: claims 1, 8 and 15 (which are rejected under the same rationale), Frazier teaches
1. A method, comprising: transmitting, from a client, a request to access image data representing an image; (“The client system 14 also includes a query routine 24 that, in response to user actions, sends requests to the server system 16 for data in the database 56… Fig. 2 shows a table 150 that is stored in the database 56… the objects stored in the table 150 are used to form “resultant” images (or images that result from a combination of other elements).”; Frazier, col. 3, lines 37-39, line 55, lines 62-64, Figs. 1-2)
Figs. 1-2 illustrate that the client sends (transmits) a query (request) to the server to access objects/images stored in table of the database.
after transmitting the request, receiving a first portion of the image data, the first portion of the image data representing a first layer, (“In response to a query… objects are extracted from the appropriate rows and columns of the table 150. The objects extracted include an object containing a background image and objects containing points, lines, and polygons to draw over the background image. Depending on what is requested, only one layer may be extracted or multiple layers may be extracted to form the resultant image… The objects extracted from the database 56 are received by the layer manager 46, which combines or aggregates the objects into a composite image (the resultant image).”; Frazier, col. 4, lines 11-17, lines 24-26, Fig. 1)
In response to the query, objects including the background image (first portion of the image data, the first portion of the image data representing a first layer) are extracted to form the resultant image (first layer), which is placed in the layer manager of the server.
(“In response to the request communicated at 204, the database system 18 extracts (at 206) objects from the table 150. The extracted objects are then sent (at 208) to the server system 16. The server system 16 combines or aggregates (at 210) the objects into a resultant image. The layer manager 46 in the server system 16 the creates (at 212) a VRML file 48 to represent the resultant image, with the VRML file 48 sent (at 214) to the client system 14.”; Frazier, col. 4, lines 46-53, Fig. 1)
Also, in response to the query (after transmitting the request), the layer manager creates a VRML file representing the resultant image (first layer) and sends this resultant image to the client (receiving a first portion of the image data, the first portion of the image data representing a first layer).
rendering a smoothed version of the first layer on a display associated with the client by performing a smoothing operation on the first layer, (“In the client system 14, the viewer routine 30 displays (at 216) the image represented by the received VRML file 28.”; Frazier, col. 4, lines 53-55, Figs. 1 and 3)
The client system displays (renders) the image represented by the VRML file (rendering... the first layer on a display associated with the client). Frazier is silent regarding the first layer being an smoothed version. However, Lee teaches this limitation.
(“Input/output circuit 160 may display the photographed real environment as a first layer and display the created AR contents as the second layer to be overlapped on the first layer. Input/output circuit 160 may display the extracted object as an object image layer to be overlapped on the second layer. For example, input/output circuit 160 may blur the first layer, overlay the object image layer on the blur first layer and the second layer.”; Lee, [0068], Fig. 2)
The input/output circuit of the user device 100 displays and blurs the first layer (rendering a smoothed version of the first layer on a display associated with the client). Frazier and Lee are silent regarding , however Agarwal teaches this limitation.
the smoothing operation being based on a parameter specifying an amount of smoothing; (“for example, some camera apps feature a “lens blur” mode that asks user to move the smart-phone while capturing multiple images. The processor of the phone estimates scene depths based on the movement and positional sensors within the phone and then blurs the image based on the estimated depth values. In some examples, controls may be provided to change the amount of blur as well as to select which depth regions to keep in focus.”; Agrawal, col. 26, lines 16-25)
Blur is applied to a captured image. The amount of blur can be adjusted by the user (parameter specifying an amount of smoothing). Agrawal is combined with Frazier and Lee such that the blurring operation of Lee can be adjusted by the user controls of Agrawal and added to the method of Frazier. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the method of Frazier by adding the feature of the smoothing operation being based on a parameter specifying an amount of smoothing, in order to improve the image quality, 3D effect, the immersive effect, and the depth of field effect, as taught by Lee ([0079]).
Frazier and Agrawal are silent regarding while rendering the smoothed version of the first layer, receiving a second portion of the image data, the second portion of the image data representing a second layer, however, Jooste and Lee teach this limitation.
while rendering the smoothed version of the first layer, receiving a second portion of the image data, the second portion of the image data representing a second layer; (“… the client display manager 100 can accomplish the layered presentation of received image blocks by dynamically adding received image blocks (e.g., updated image information) as image elements to a currently presented HTML document, and then modifying style attributes of the added image elements to control their presentation in a layered manner.”; Jooste, [0020], Fig. 2)
Fig. 2 illustrates the layered presentation of received image blocks (image layers).
(“In this example, block 110 serves as a key frame, in that it represents, for example, an entire user interface screen when it is first opened by a user. Key frames may be also transferred from the server system 122 to the client display manager 100 at subsequent times… Subsequently received blocks 111-114 represent modifications or updates to portions of the key frame and are respectively layered upon one another by the client display manager 100.”; Jooste, [0022], Fig. 2)
Fig. 2 illustrates that image block 110 is the background image (first layer) and that image blocks 111-114 are layered on top of the background image.
(“Figs. 3A-3F are each visualizations of a layering data structure used by the client display manager 100 to layer the received image blocks 110-114 described above… Each of Figs. 3A-3E show the layering data structure after a respective one of image blocks 110-114 has been received by the client display manager 100 and added to the layering data structure. In particular Fig. 3A shows image block 110 assigned to layer 0 at a first time… Fig. 3B shows image blocks 110 and 111 respectively assigned to layers 0 and 1 at a second time… Fig. 3C shows image blocks 110-112 respectively assigned to layers 0-2 at a third time… ”; Jooste, [0023], [0024], [0026], [0028], Figs. 2, 3A-3E)
Fig. 3A illustrates the background image block 110 is received and displayed (rendered). Fig. 3B illustrates that, while the background image block 111 is being displayed (while rendering… the first layer), image block 111 is received (receiving a second portion of the image data) and displayed as a layer on background image 110 (the second portion of the image data representing a second layer). Frazier and Jooste are silent regarding the first layer being a smoothed version. However, Lee teaches this limitation.
(“Input/output circuit 160 may display the photographed real environment as a first layer and display the created AR contents as the second layer to be overlapped on the first layer. Input/output circuit 160 may display the extracted object as an object image layer to be overlapped on the second layer. For example, input/output circuit 160 may blur the first layer, overlay the object image layer on the blur first layer and the second layer.”; Lee, [0068], Fig. 2)
The input/output circuit of the user device 100 displays and blurs the first layer (rendering the smoothed version of the first layer). Lee is combined with Frazier, Agrawal and Jooste such that the background portion of the image (first layer) of Jooste is rendered as blurred (smoothed) background of Lee. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the method of Frazier by adding the feature of while rendering the smoothed version of the first layer, receiving a second portion of the image data, the second portion of the image data representing a second layer, in order to present a dynamically changing image on a client device without use of, or with limited access to, a frame buffer for displaying image data, as taught by Jooste ([0013]) and in order to significantly and efficiently enhance the 3D effect, the immersive effect, and the depth of field effect in real-time image processing, as taught by Lee ([0070]).
Frazier and Agrawal are silent regarding rendering the second layer on the display while displaying the smoothed version of the first layer on the display, however, Jooste and Lee teach this limitation.
and rendering the second layer on the display while displaying the smoothed version of the first layer on the display. (“… the client display manager 100 can accomplish the layered presentation of received image blocks by dynamically adding received image blocks (e.g., updated image information) as image elements to a currently presented HTML document, and then modifying style attributes of the added image elements to control their presentation in a layered manner.”; Jooste, [0020], Fig. 1)
A layered image is created by dynamically adding image blocks as layers in the image. Fig. 1 illustrates the image layers being received, where block 110 is the background (first layer).
(“Fig. 2 shows blocks 110-114 described with reference to Fig. 1 presented upon display 121. In this example, block 110 serves as a key frame, in that it represents, for example, an entire user interface screen when it is first opened by a user… Key frames may be transferred from the server system 122 to the client display manager 100 at subsequent times as described below. Subsequently received blocks 111-114 represent modifications or updates to portions of the key frame and are respectively layered upon one another by the client display manager 100.”; Jooste, [0022], Fig. 2)
Fig. 2 illustrates the layered image, where block 110 is the background (first layer) and blocks 111-114 are additional layers that have been added, at subsequent times, while the background 110 is displayed. Image block 111 is considered to be, for example, the second layer (rendering the second layer on the display while displaying... first layer on the display)
(“Each of Figs 3A-3E show the layering data structure after a respective one of image blocks 110-114 has been received by the client display manager 100 and added to the layering data structure. In particular, Fig. 3A shows image block 110 assigned to layer 0 at a first time… Fig. 3B shows image blocks 110 and 111 respectively assigned to layers 0 and 1 at a second time… Fig. 3C shows image blocks 110-112 respectively assigned to layers 0-2 at a third time… Fig. 3D shows image blocks 110-113 respectively assigned to layers 0-3 at a fourth time… Fig. 3E shows image blocks 110-114 respectively assigned to layers 0-4 at a fifth time… ”; Jooste, [0024], [0026], [0028], [0030], [0032], Figs. 3A-3D)
Fig. 3A illustrates the background 110 being received and displayed. Fig. 3B illustrates the second layer 111 being added to the background image (rendering the second layer on the display while displaying the... first layer on the display). Jooste is silent regarding the first layer being smoothed, however, Lee teaches this limitation.
(“For example, AR device 100 may make the first image layer blur and overlap the object image layer on the blurred first image layer in order to improve a 3D effect and depth of field in real time.”; Lee, [0053])
The first image layer is blurred (smoothed version of the first layer) and the object image layer (rendering the second layer on the display) is overlapped onto the first layer (while displaying the smoothed version of the first layer on the display). Jooste and Lee are combined with Frazier and Agrawal such that the background layer of Jooste is the blurred first layer of Lee. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the method of Frazier by adding the feature of rendering the second layer on the display while displaying the smoothed version of the first layer on the display, in order to present a dynamically changing image on a client device without use of, or with limited access to, a frame buffer for displaying image data, as taught by Jooste ([0013]) and in order to improve a 3D effect and depth of field in real time, as taught by Lee ([0053]).
Claim 8 is a computer program product analogous to the method of claim 1, is similar in scope and is rejected under the same rationale. Claim 8 has an additional limitation. Re: claim 8, Frazier teaches
8. A computer program product comprising a nontransitive storage medium, the computer program product including code that, when executed by processing circuitry of a computing device, causes the processing circuitry to perform a method, the method comprising: (“The storage units referred to in this discussion include one or more machine-readable storage media for storing data and instructions. The storage media include different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; and optical media such as compact disks (CDs) or digital video disks (DVDs). Instructions that make up the various software routines or modules are stored in respective storage units. The instructions when executed by a respective control unit cause the corresponding system to perform programmed acts.”; Frazier, col. 5, lines 47-61)
Machine-readable storage media (storage medium) stores instructions (including code) that when executed by a control unit (executed by processing circuitry of a computing device) causes the system to perform acts (causes the processing circuity to perform a method).
Claim 15 is an apparatus analogous to the method of claim 1, is similar in scope and is rejected under the same rationale. Claim 15 has an additional limitation. Re: claim 15, Frazier teaches
15. An electronic apparatus, the electronic apparatus comprising: memory; and a processor coupled to the memory, the processor being configured to: (“Fig. 1 illustrates a communications system 10 having a data network 12 that is coupled to a client system 14 and a server system 16. ”; Frazier, col. 2, lines 9-11, Fig. 1)
Fig. 1 illustrates a storage unit 38 (memory) and a control unit 38 (processor coupled to the memory).
Re: claims 2, 9 and 16 (which are rejected under the same rationale), Frazier, Lee, Agrawal and Jooste teach
2. The method as in claim 1, wherein the smoothing operation is performed on a graphics processing unit of the client.
(“Referring to FIG. 1, a multilayered augmented reality image (e.g., multilayered AR image or multilayered AR video) may be created using augmented reality (AR) user device 100, server 200, and various displays 410, 420, and 430... AR user device 100 may be connected with server 200 through network 300...”; Lee, [0046], Fig. 1)
Fig. 1 illustrates that the a multilayered AR image is created using an AR user device (client), a server and multiple displays. The AR user device (client) is connected to the server over network 300.
(“The image processor may be configured to use a Gaussian filter to blur the background image layer... For example, AR user device 100 may make the first image layer blur and overlap the object image layer on the blurred first image layer in order to improve a 3D effect and depth of field in real time.”, Lee, [0013], [0053], Fig. 2)
Fig. 2 illustrates the AR user device (client), that makes the first image layer blur. The image processor, of the AR user device, uses a Gaussian filter to blur the background image layer. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the method of Frazier by adding the feature of the smoothing operation is performed on a graphics processing unit of the client, in order to improve a 3D effect and depth of field in real time, as taught by Lee ([0053]).
Re: claims 4, 11 and 18 (which are rejected under the same rationale), Frazier, Lee, Agrawal and Jooste teach
4. The method as in claim 1, wherein the smoothed version of first layer includes a first set of features, (“the background image layer may be processed to be blur after extracting at least one high priority object therefrom, and in order to efficiently enhance a 3D effect and depth of filed by rendering the blur.”; Lee, [0044])
The background image layer may be processed to be blur (smoothed version of first layer includes a set of features), where the features are considered to be the blur with at least one object extracted.
and wherein the method further comprises: rendering, as the second layer, a second set of features over the first set of features. (“For example, AR user device 100 may make the first image layer blur and overlap the object image layer on the blurred first image layer in order to improve a 3D effect and depth of filed in real time.”; Lee, [0053])
The AR user device makes the first image layer blur and overlaps the object image layer (rendering, as a second layer, a second set of features over the first set of features) on the blurred first image layer. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the method of Frazier by adding the feature of the method further comprises: rendering, as the second layer, a second set of features over the first set of features, in order to improve a 3D effect and depth of field in real time, as taught by Lee ([0053]).
Claim(s) 3, 10 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Frazier, Lee, Agrawal and Jooste as applied to claims 1, 8 and 15 above, and further in view of Konrad et al. U.S. Pub. No. 2022/0020132.
Re: claims 3, 10 and 17 (which are rejected under the same rationale), Frazier, Lee and Agrawal are silent regarding the smoothed version of the first layer is rendered over a set of image pixels of the image, wherein the first layer includes a first set of features, however, Jooste and Kondrad teach
3. The method as in claim 1, wherein the smoothed version of the first layer is rendered over a set of image pixels of the image, wherein the first layer includes a first set of features, (“… the client display manager 100 can accomplish the layered presentation of received image blocks by dynamically adding received image blocks (e.g., updated image information) as image elements to a currently presented HTML document, and then modifying style attributes of the added image elements to control their presentation in a layered manner.”; Jooste, [0020], Fig. 1)
A layered image is created by dynamically adding image blocks as layers in the image. Fig. 1 illustrates the image layers being received, where block 110 is the background (first layer is rendered over a set of image pixels of the image).
(“the manipulation of the layers is performed by generating a blurred version of the initial partial image and blending the initial partial image with the blurred version so as to reduce perceived artefacts caused by perspective displacement of the initial partial image relative to one or more of the other initial partial images.”; Kondrad, [0053])
An image includes plural layers, where, the initial partial image (first layer) is blurred (smoothed version of the first layer) and then blended with initial partial image. The set of features is considered to be the blurring of the image Kondrad and Jooste are combined with Frazier, Lee, Agrawal and Jooste such that the blurred layer of Kondrad is rendered over the layers of Jooste. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the method of Frazier by adding the feature of the smoothed version of the first layer is rendered over a set of image pixels of the image, wherein the first layer includes a first set of features, in order to present a dynamically changing image on a client device without use of, or with limited access to, a frame buffer for displaying image data, as taught by Jooste ([0013]) and in order to reduce perceived artifacts caused by perspective displacement of the initial partial image relative to other initial partial images, as taught by Kondrad ([0053]).
Frazier, Lee, Agrawal and Jooste teach wherein the second portion of the image data further includes a third layer, (“Each of Figs 3A-3E show the layering data structure after a respective one of image blocks 110-114 has been received by the client display manager 100 and added to the layering data structure. In particular, Fig. 3A shows image block 110 assigned to layer 0 at a first time… Fig. 3B shows image blocks 110 and 111 respectively assigned to layers 0 and 1 at a second time… Fig. 3C shows image blocks 110-112 respectively assigned to layers 0-2 at a third time… Fig. 3D shows image blocks 110-113 respectively assigned to layers 0-3 at a fourth time… Fig. 3E shows image blocks 110-114 respectively assigned to layers 0-4 at a fifth time… ”; Jooste, [0024], [0026], [0028], [0030], [0032], Figs. 3A-3D)
Fig. 3A illustrates the background 110 being received and displayed. Figs. 3B-3C illustrate additional layers 111-112, where layer 111 is considered to be the second layer and layer 112 is considered to be the third layer (the second portion of the image data further includes a third layer) that are added to the background image.
Figs. 3A-3C illustrate the image being layered, where block 110 is the background (first layer) and blocks 111-114 are additional layers that are added, at subsequent times, to the background 110.
and wherein the method further comprises: rendering, as the second layer, a second set of features over a first subset of the set of image pixels; and rendering, as the third layer, a third set of features over a second subset of the set of image pixels. (“Each of Figs 3A-3E show the layering data structure after a respective one of image blocks 110-114 has been received by the client display manager 100 and added to the layering data structure. In particular, Fig. 3A shows image block 110 assigned to layer 0 at a first time… Fig. 3B shows image blocks 110 and 111 respectively assigned to layers 0 and 1 at a second time… Fig. 3C shows image blocks 110-112 respectively assigned to layers 0-2 at a third time… Fig. 3D shows image blocks 110-113 respectively assigned to layers 0-3 at a fourth time… Fig. 3E shows image blocks 110-114 respectively assigned to layers 0-4 at a fifth time… ”; Jooste, [0024], [0026], [0028], [0030], [0032], Figs. 2 and 3A-3D)
Figs. 3A-3C illustrate the rendering of additional layer 111 (second layer) and additional layer 112 (third layer). Fig. 2 illustrates the additional layer 111 (second layer), which includes a second set of image features over background layer 110 (a first subset of the set of image pixels). Fig. 2 also illustrates additional layer 112 (third layer), which includes a third set of image features over a portion of additional layer 111 and the background 110 (over a second subset of the set of image pixels). Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the method of Frazier by adding the feature of the second portion of the image data further includes a third layer, and wherein the method further comprises: rendering, as the second layer, a second set of features over a first subset of the set of image pixels; and rendering, as the third layer, a third set of features over a second subset of the set of image pixels, in order to present a dynamically changing image on a client device without use of, or with limited access to, a frame buffer for displaying image data, as taught by Jooste. ([0013])
Claim(s) 5, 6, 12, 13, 19 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Frazier, Lee, Agrawal and Jooste as applied to claims 4, 11 and 18 above, and further in view of Konrad and Barenburg et al. U.S. Pub. No. 2011/0169823.
Re: claims 5, 12 and 19 (which are rejected under the same rationale), Frazier, Lee and Agrawal are silent regarding the smoothed version of the first layer is rendered over a set of image pixels of the image, however, Jooste and Kondrad teach
5. The method as in claim 4, wherein the smoothed version of the first layer is rendered over a set of image pixels of the image, (“… the client display manager 100 can accomplish the layered presentation of received image blocks by dynamically adding received image blocks (e.g., updated image information) as image elements to a currently presented HTML document, and then modifying style attributes of the added image elements to control their presentation in a layered manner.”; Jooste, [0020], Fig. 1)
A layered image is created by dynamically adding image blocks as layers in the image. Fig. 1 illustrates the image layers being received, where block 110 is the background (first layer is rendered over a set of image pixels of the image).
(“the manipulation of the layers is performed by generating a blurred version of the initial partial image and blending the initial partial image with the blurred version so as to reduce perceived artefacts caused by perspective displacement of the initial partial image relative to one or more of the other initial partial images.”; Kondrad, [0053])
An image includes plural layers, where, the initial partial image (first layer) is blurred (smoothed version of the first layer) and then blended with initial partial image. Kondrad and Jooste are combined with Frazier, Lee, Agrawal and Jooste such that the blurred layer of Kondrad is rendered over the layers of Jooste. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the method of Frazier by adding the feature of the smoothed version of the first layer is rendered over a set of image pixels of the image, in order to present a dynamically changing image on a client device without use of, or with limited access to, a frame buffer for displaying image data, as taught by Jooste ([0013]) and in order to reduce perceived artifacts caused by perspective displacement of the initial partial image relative to other initial partial images, as taught by Kondrad ([0053]).
Frazier, Lee, Agrawal and Jooste and Kondrad are silent regarding rendering the second set of features includes: performing a crossfading operation on a subset of the set of image pixels, however, Barenburg teaches
and wherein rendering the second set of features includes: performing a crossfading operation on a subset of the set of image pixels. (“… the cross fade may be achieved by gradually changing the transparency value for the images from a value corresponding to full opacity for the first image layer to one corresponding to full transparency. Thus, a smooth transition and cross fade from the initial image sequence to be subsequent image sequences is achieved.”; Barenburg, [0080])
The cross fading operation is performed by gradually changing the transparency value for the images from full opacity to full transparency (performing crossfading operation on a subset of the set of image pixels). Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the method of Frazier by adding the feature of rendering the second set of features includes: performing a crossfading operation on a subset of the set of image pixels, in order to achieve a smooth transition and cross fade from the initial image sequence to the subsequent image sequence, as taught by Barenburg. ([0080])
Re: claims 6, 13 and 20 (which are rejected under the same rationale), Frazier, Lee, Agrawal and Jooste are silent regarding , however, Konrad teaches this limitation.
6. The method as in claim 5, wherein the crossfading operation includes a convolution operation with a specified blurring kernel. (“… the generating the blurred version of the initial partial image comprises applying a blurring filter to the initial partial image… the kernel size of the blurring filter may be based on the size of the perspective displacement of the initial partial image relative to one or more of the other initial partial images. The buffer the kernel size the more blurring spread may be applied to each pixel of the initial partial image. As a result, a visual mist is created in the blurred version of the initial partial image extending beyond the edges of the initial partial image… the blurring may be performed by filtering or crossfading the initial partial images.”; Konrad, [0016], [0017])
Blurring may be performed by crossfading the initial partial images. Blurring is performed by applying a blurring filter to each pixel of the partial image (convolution operation), where the kernel size of the blurring filter (specified blurring kernel) is based on the size of the perspective displacement of the initial partial image relative to other initial partial images. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the method of Frazier by adding the feature of the crossfading operation includes a convolution operation with a specified blurring kernel, in order to provide a plurality of enhanced partial images which together represent an enhanced 3D image with less viewable gaps and stripes, as taught by Konrad. ([0010]).
Claim(s) 7, 14 and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Frazier, Lee, Agrawal, Jooste, Kondrad and Barenburg as applied to claims 5, 12 and 19 above, and further in view of Rieder et al. U.S. Pub. No. 2002/0196369.
Re: claims 7, 14 and 21 (which are rejected under the same rationale), Frazier, Lee, Agrawal, Jooste, Kondrad and Barenburg are silent regarding the crossfading operation is applied to subblocks of blocks of pixels, the subblocks located at a corner of the blocks, however, Rieder teaches this limitation.
7. The method as in claim 5, wherein the crossfading operation is applied to subblocks of blocks of pixels, the subblocks located at a corner of the blocks. (“Fig. 1 is a schematic illustration of a combined picture which has, within a displayable screen region G, a first image M in a first active region A1 and a second image S in a second active region A2, the first and second active regions A1, A2 having an overlap region A12 in which the first and second images M, S overlap. A background image B is displayed in the area outside the first and second active regions A1, A2…”; Rieder, [0034], [0051], Fig.1)
Fig. 1 illustrates a combined picture of a background image B, with a first image M (first block of pixels) and a second image S (second block of pixels) layered on top of the background. The first and second images overlap in overlap region MS (which includes a subblock of the first block of pixels and a subblock of the second block of pixels). This overlap region MS (subblocks of the blocks of pixels) is located at a corner for both blocks of pixels.
(“Initially, first weighting factor m1 is one, causing the video information of the first image M to be displayed in overlap region A12, while first image M is in the foreground. Starting from the first image M in the foreground for the combined picture shown at the top, a cross-fading takes place to the second image S located in the foreground as indicated in the last combined picture shown at the bottom.”; Rieder, [0051], Fig. 9)
Fig. 9 illustrates cross-fading between the first image M and the second image S overlapping at the corner MS. The top image shows the first image M (including the overlapping portion) in the foreground. Then, cross-fading is performed, and the bottom image shows the second image S (including the overlapping portion) in the foreground (crossfading operation is applied to subblocks of the blocks of pixels, the subblocks located at a corner of the blocks). Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the method of Frazier by adding the feature of the crossfading operation is applied to subblocks of blocks of pixels, the subblocks located at a corner of the blocks, in order to display image information from a first and second image within the overlap region such that, crossfading from the first image to the second image is possible when the foreground image changes, as taught by Rieder. ([0007])
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DONNA J RICKS whose telephone number is (571)270-7532. The examiner can normally be reached on M-F 7:30am-5pm EST (alternate Fridays off).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona Faulk can be reached on 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Donna J. Ricks/Examiner, Art Unit 2618
/DEVONA E FAULK/Supervisory Patent Examiner, Art Unit 2618
Prosecution Insights