Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
Claim(s) 1-20 rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1-20 of U.S. Patent No. 12,266,066
Although the claims at issue are not identical, they are not patentably distinct from each other because ‘197 is broader in scope than ‘066 and all limitations are present in a corresponding claim.
Table 1 below shows an example claim mapping between the present application and U.S. Patent No. 12,266,066
Table 1. Example Claim Mapping
19/062,197
12,266,066
1. A mobile device including:
a viewing area configured to present overlay images to a user over a scene in the viewing area;
at least one sensor configured to determine a location and an orientation of the mobile device;
a processor coupled to the mobile device;
a memory accessible to the processor; and
programming in the memory, wherein execution of the programming by the processor configures the mobile device to perform functions, including functions to:
monitor the location and the orientation of the mobile device using the at least one sensor;
request a previously captured image corresponding to where the previously captured image was captured in relation to the location;
receive the requested previously captured image;
generate an overlay image from the received previously captured image; and
display the overlay image in the viewing area.
1. A mobile device including:
an optical assembly, the optical assembly having a viewing area for viewing a scene and configured to present overlay images to a user over the scene in the viewing area;
at least one sensor configured to determine a location and an orientation of the mobile device;
a processor;
a memory accessible to the processor; and
programming in the memory, wherein execution of the programming by the processor configures the mobile device to perform functions, including functions to:
monitor the location and the orientation of the mobile device using the at least one sensor;
request from a server previously captured images corresponding to where the previously captured images were captured in relation to the location;
receive the requested previously captured images from the server;
select images from the requested previously captured images responsive to the location and the orientation of the mobile device;
aggregate selected images corresponding to the location within a location range associated with a visual cue;
generate overlay images from the aggregated selected images including an image icon corresponding to most recent content within the aggregated selected images; and
present the overlay images on the optical assembly including the image icon in the viewing area.
Claim(s) 1-20 rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1-20 of U.S. Patent No. 11,948,262
Although the claims at issue are not identical, they are not patentably distinct from each other because ‘197 is broader in scope than ‘262 and all limitations are present in a corresponding claim.
Table 1 below shows an example claim mapping between the present application and U.S. Patent No. 11,948,262
Table 1. Example Claim Mapping
19/062,197
11,948,262
1. A mobile device including:
a viewing area configured to present overlay images to a user over a scene in the viewing area;
at least one sensor configured to determine a location and an orientation of the mobile device;
a processor coupled to the mobile device;
a memory accessible to the processor; and
programming in the memory, wherein execution of the programming by the processor configures the mobile device to perform functions, including functions to:
monitor the location and the orientation of the mobile device using the at least one sensor;
request a previously captured image corresponding to where the previously captured image was captured in relation to the location;
receive the requested previously captured image;
generate an overlay image from the received previously captured image; and
display the overlay image in the viewing area.
1. A mobile device including:
an optical assembly, the optical assembly having a viewing area for viewing a scene and configured to present overlay images to a user over the scene in the viewing area;
at least one sensor configured to determine a location and an orientation of the mobile device;
a processor;
a memory accessible to the processor; and
programming in the memory, wherein execution of the programming by the processor configures the mobile device to perform functions, including functions to:
monitor the location and the orientation of the mobile device using the at least one sensor;
request from a server previously captured images corresponding to where the previously captured images were captured in relation to the location;
receive the requested previously captured images from the server;
select images from the requested previously captured images responsive to the location and the orientation of the mobile device; generate overlay images from the selected images including image icons associated with the selected images;
present the overlay images on the optical assembly;
receive an image selection identifying one of the image icons in the presented overlay images;
display, via the optical assembly, the selected image associated with the identified image icon in the viewing area; and
alter the identified image icon on the optical assembly based on at least one of (a) distance of the mobile device to a visual cue within at least one of the images corresponding to the location or (b) quantity of image content relating to the visual cue, whereby the icon is reduced in size the further the visual cue is from the mobile device and the icon is increased in size for larger quantity of image content relating to the visual cue.
Claim(s) 1-20 rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1-20 of U.S. Patent No. 11,176,751
Although the claims at issue are not identical, they are not patentably distinct from each other because ‘197 is broader in scope than ‘751 and all limitations are present in a corresponding claim.
Table 1 below shows an example claim mapping between the present application and U.S. Patent No. 11,176,751
Table 1. Example Claim Mapping
19/062,197
11,176,751
1. A mobile device including:
a viewing area configured to present overlay images to a user over a scene in the viewing area;
at least one sensor configured to determine a location and an orientation of the mobile device;
a processor coupled to the mobile device;
a memory accessible to the processor; and
programming in the memory, wherein execution of the programming by the processor configures the mobile device to perform functions, including functions to:
monitor the location and the orientation of the mobile device using the at least one sensor;
request a previously captured image corresponding to where the previously captured image was captured in relation to the location;
receive the requested previously captured image;
generate an overlay image from the received previously captured image; and
display the overlay image in the viewing area.
1. A mobile device including:
an optical assembly, the optical assembly having a viewing area for viewing a scene and configured to present overlay images to a user over the scene in the viewing area; and at
least one sensor configured to determine a location and an orientation of the mobile device;
a processor coupled to the mobile device;
a memory accessible to the processor; and
programming in the memory, wherein execution of the programming by the processor configures the mobile device to perform functions, including functions to:
monitor the location and the orientation of the mobile device using the at least one sensor;
request from a server previously captured images corresponding to where the previously captured images were captured in relation to the location;
receive the requested previously captured images from the server;
select images from the requested previously captured images responsive to the location and the orientation of the mobile device; generate overlay images from the selected images including image icons associated with the selected images;
present the overlay images on the optical assembly;
receive an image selection identifying one of the image icons in the presented overlay images; and
display, via the optical assembly, the selected image associated with the identified image icon in the viewing area.
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Use of indicates a limitation is not explicitly disclosed by the reference alone.
Claim(s) 1-2, 7-11, 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ishikawa (US Patent 10,878,629) in view of Yu (US Patent 9,288,385)
Claim 1
Ishikawa discloses a mobile device including:
a viewing area configured to present overlay images to a user over a scene in the viewing area (Ishikawa, Col. 6: "Thus, the HMD 10 can provide a field of view, in which an image associated with objects distributed in a reality space is superimposed on the objects, to the user wearing the HMD 10 via the see-through display. Hereinafter, an image associated with an object is referred to as an AR icon.");
at least one sensor configured to determine a location and an orientation of the mobile device (Ishikawa, Col. 10-11: "When the user changes the orientation of the head, i.e., the display unit 11 (head track), the range of theta. in the field of view V");
a processor coupled to the mobile device;
a memory accessible to the processor (Fig. 3; CPU); and
programming in the memory, wherein execution of the programming by the processor configures the mobile device to perform functions, including functions to (Fig. 3; Memory; “program for executing control…”):
monitor the location and the orientation of the mobile device using the at least one sensor (Ishikawa, Col. 6: "Thus, the HMD 10 can provide a field of view, in which an image associated with objects distributed in a reality space is superimposed on the objects, to the user wearing the HMD 10 via the see-through display. Hereinafter, an image associated with an object is referred to as an AR icon.");
request a previously captured image corresponding to where the previously captured image was captured in relation to the location (Ishikawa, Col. 21: "For example, the AR server 40 stores a photography spot database as the object database 404. Image data photographed by a digital camera or a portable information terminal (hereinafter, referred to as picture data) are uploaded to a photo-sharing site on the Internet 50, a blog on the Internet 50, or the like. The AR server 40 acquires photographic position information that is added to those pieces of picture data as metadata. The photographic position information is, for example, acquired at photography by a photographic apparatus including the GPS communication unit 305. The AR server 40 holds the acquired photographic position information as the photography spot database");
Ishikawa does not explicitly disclose, but
PNG
media_image1.png
479
619
media_image1.png
Greyscale
receive the requested previously captured image (610; Yu: “610, the receiving module 210 of the GUI generating program 129-a in the image playback device 120-a utilizes the communication circuit 123-a to receive the photographing positions distribution information, the multiple characteristic parameter sets, and the multiple thumbnails transmitted from the image providing server 110 through the Internet 130.”) ;
generate an overlay image from the received previously captured image (GUI generation); and
display the overlay image in the viewing area (display of thumbnails).
PNG
media_image2.png
482
632
media_image2.png
Greyscale
“The GUI 800 comprises a rectangular area 810, a reference indicator 820, and multiple option objects (e.g., the example option objects 831˜835 shown in FIG. 8). In the aforementioned operation 612, the option object generating module 220 may utilize the control circuit 127-a to respectively utilize the multiple thumbnails to be main parts of the multiple option objects 831˜835, so as to directly show the thumbnails of corresponding image data on the option objects 831˜835.“
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to receive requested images.
As suggested by Ishikawa, a photo recommendation service is a considered application, and one of ordinary skill in the art would have motivation to provide improved display as disclosed by Yu: (“In this situation, the traditional image list is especially incapable of assisting the user to effectively find the image of interest, thereby deteriorating the aforementioned problem and adversely affecting the user satisfaction level of the image sharing websites.”) One of ordinary skill in the art would have had a reasonable expectation of success because both references consider retrieval of nearby POIs and use in a photo selection/recommendation service.
Claim 2
Ishikawa discloses wherein the mobile device is an eyewear device comprising:
PNG
media_image3.png
320
524
media_image3.png
Greyscale
a frame supporting an optical assembly having the viewing area (housing 112); and
a temple extending from a lateral side of the frame (e.g. 11);
wherein the optical assembly is a see-through optical assembly supported by the frame (“The field of view V is formed on the see-through display. In the example of the figure, two buildings in the reality space are displayed in a see-through manner”).
Claim 7
Ishikawa discloses wherein execution of the programming by the processor further configures the mobile device to perform additional functions including functions to: determine a field of view of the optical assembly; calculate a location coordinate range within the field of view for a determined orientation of the mobile device; and request the previously captured image whereby the previously captured image has location coordinates within the calculated location coordinate range (“"shown in A of FIG. 7, coordinates of a reference point of the field of view V on the cylindrical coordinates CO are assumed as (.theta.v, hv), and coordinates of a reference point of an object A positioned in the field of view V are assumed as (.theta.a, ha). The reference points of the field of view V and the object A may be set at any points and, in this example, set at the upper left corner of the field of view V having a rectangular shape and the object A. alpha.| [] is a width angle of the field of view V in the cylindrical coordinates, and a value thereof is determined depending on the design or specifications of the display unit 11")
Claim 8
Ishikawa discloses wherein execution of the programming by the processor further configures the mobile device perform additional functions including functions to:
PNG
media_image4.png
685
593
media_image4.png
Greyscale
monitor a field of view of the optical assembly (Fig. 5; V);
calculate location coordinates within the field of view for a determined orientation of the mobile device; map previously captured images to the calculated location coordinates; ("In the example shown in FIG. 10, the positions of the objects A11 to A14 at local coordinates are calculated as local coordinates (xa11, ya11), (xa12, ya12), (xa13, ya13), and (xa14, ya14), respectively")
identify at least one visual cue; and position at least one image icon within the overlay image responsive to the calculated location coordinates and the at least one visual cue (“(Col. 15: "In the example shown in FIG. 10, the group management unit 25 groups the four objects A11 to A14 in the field of view V as a group G10, the four objects A11 to A14 having associated additional information of an "eating place". In this case, the associated additional information is an attribute of the objects. Note that in this example of operation the group management")
Claim 9
Ishikawa discloses wherein the optical assembly further comprises a camera and wherein execution of the programming by the processor further configures the mobile device to perform additional functions including functions to: capture an image with the camera; obtain a location of the mobile device at the time of capture of the image; associate the location at the time of capture with the image; and send the image with the associated location to a server for storage as a previously captured image for the location of the mobile device at the time of capture of the image (Col. 21: "For example, the AR server 40 stores a photography spot database as the object database 404. Image data photographed by a digital camera or a portable information terminal (hereinafter, referred to as picture data) are uploaded to a photo--sharing site on the Internet 50, a blog on the Internet 50, or the like. The AR server 40 acquires photographic position information that is added to those pieces of picture data as metadata. The photographic position information is, for example, acquired at photography by a photographic apparatus including the GPS communication unit 305. The AR server 40 holds the acquired photographic position information as the photography spot database Additionally, in a case where there is a statistically significant season among photography seasons in photography spots out of the extracted photography spots, the group management unit 25 can group photography spots that are processed in the moment close to the significant photography season. The photography season can be identified from the metadata added to the picture data.")
Claim 10
The same teachings and rationales in claim 1 are appliable to claim 10.
Claim 11
The same teachings and rationales in claim 2 are appliable to claim 11.
Claim 16
The same teachings and rationales in claim 7 are appliable to claim 16.
Claim 17
The same teachings and rationales in claim 8 are appliable to claim 17.
Claim 18
The same teachings and rationales in claim 9 are appliable to claim 18.
Claim 19
Examiner’s Interpretation:
Machine readable media can encompass forms of signal transmission media that falls outside of the four statutory categories of invention. MPEP 2106; citing In re Nuijten, 500 F.3d 1346, 84 USPQ2d 1495 (Fed. Cir. 2007). A claim whose BRI covers both statutory and non-statutory embodiments embraces subject matter that is not eligible for patent protection and therefore is directed to non-statutory subject matter. MPEP 2106.
Claims 19-20 as drafted recites A non-transitory computer-readable medium
The broadest reasonable interpretation of the claimed medium in view of Applicant’s specification covers only eligible subject matter.
Claim Mapping:
The same teachings and rationales in claim 1 are appliable to claim 19.
Claim 20
The same teachings and rationales in claim 7 are appliable to claim 20.
Claim(s) 4, 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ishikawa (US Patent 10,878,629) in view of Yu (US Patent 9,288,385) and Ballard (US 2016/0048211)
Claim 4
Ishikawa does not disclose, but Ballard discloses wherein execution of the programming by the processor further configures the mobile device to perform additional functions including functions to: monitor for an image termination instruction on the eyewear device; and cancel presentation of the displayed overlay image responsive to the image termination instruction (e.g. Figs. 16A-B showing menu driven enabling and disabling of AR display)
Before the effective date of this application, it would have been obvious to one of ordinary skill in the art to use hands-free input to control the display. The motivation would have been to allow user control of what is displayed.
Claim 13
The same teachings and rationales in claim 4 are appliable to claim 13.
Allowable Subject Matter
Claim(s) 3, 5, 6, 12, 14, 15 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Additional Prior Art
Additional prior art relevant to Applicant’s disclosure but not relied upon:
Bailang (US 2014/0317511) also considers a photography overlay.
PNG
media_image5.png
467
611
media_image5.png
Greyscale
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN M GRAY whose telephone number is (571)272-4582. The examiner can normally be reached on Monday through Friday, 9:00am-5:30pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached on (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN M GRAY/Primary Examiner, Art Unit 2611
Prosecution Insights