Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Terminal Disclaimer
The terminal disclaimer filed on 8/4/2026 disclaiming the terminal portion of any patent granted on this application which would extend beyond the expiration date of patent 11,069,138 has been reviewed and is accepted. The terminal disclaimer has been recorded, and the previous double patenting rejection has been withdrawn.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8, 10-14 and 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Douris et al. (Pub. No. US 2010/0023878), in view of Latta et al. (Pub. No. US 2014/0306891), and further in view of Kokkas et al. (Pub. No. US 2011/0153198).
Regarding claim 1, Douris discloses a method, comprising:
determining a geographical location of a user device (Par. [0101]: “GPS module 2324 is configured to determine location information according to GPS techniques. For example, GPS module 2324 may include a GPS receiver configured to receive signals from four or more GPS satellites, and to determine a position and heading of electronic device 2300 using trilateration”);
determining that the geographical location of the user device matches the digital element based on a distance difference between the geographical location of the user device and a geographical location of the digital element (Par. [0068]: “In another embodiment, second user 1002 may interact with entity 106 by entering a region that is proximate to entity 106 (e.g., moving to within a predetermined distance of entity 106), or by entity 106 entering a region proximate to second user 1002 (if entity 106 is mobile). In any case, due to the interaction with entity 106 (assuming any access attributes associated with virtual note 108 allow for it), a display of electronic device 1004 may display virtual representation 602 of virtual note 108”. In particular, virtual note 108 is a digital element created by a first user and attached to a physical entity 106 (e.g. a building) at a particular geographical location, as disclosed in [0056]. When the second user’ device moves to within a predetermined distance of the physical entity 106, it could be said that the geographical location of the second user’s device matches the virtual note 108 based on a distance difference between the geographical location of the second user’s device and a geographical location of the virtual note);
in response to the determination that the geographical location of the user device matches the digital element, providing an indication that the digital element is selectable (Par. [0128]: “FIG. 30 shows an example electronic device 3000 having a display 3002 that shows an alert message 3004, according to an example embodiment of the present invention. Electronic device 3000 may display alert message 3004 in response to receiving alert signal 2610 from notes engine 2210. In the example of FIG. 30, alert message 3004 includes the textual message "Your network have left 5 notes in the vicinity." For instance, with regard to alert message 3004, location comparator 2604 may have determined five of virtual notes 2500a-2500n that have entities proximate to a determined location of electronic device 3000. Alert message 3004 may have any configuration, including any configuration of text, graphics, and/or audio, to inform a user of electronic device 3000 that one or more virtual notes are nearby”. In particular, an alert message, e.g. a text, graphics, or audio message, is an indication that a virtual note is selectable. Note that claim 13 of the present application further clarifies that “providing the indication that the digital element is selectable includes providing a visual indication”);
providing to a user a representation of the digital element without providing a content of the digital element obtainable by the user, including by providing adigital element that indicates a (Fig. 11 – virtual representation 602, and par. [0069]: “As shown in FIG. 11, display 1100 shows virtual representation 602 in a virtual space superimposed on a region similar to the region of image 302 shown in FIG. 3”. In particular, virtual representation 602 is a visual representation of virtual note 108. Note that at the time shown in Fig. 11, the entire content of virtual note 108, which may include text, graphics, audio and video content, is yet to be rendered. The virtual note 108 is a digitally obtained object located at a real world location because it can be obtained by electronic device 1004 when it moves to within a predetermined distance from the real world location to which the virtual note 108 is attached);
receiving a user selection indication that selects the representation of the digital element to obtain the content of the digital element (Par. [0071]: “In step 904, the virtual representation of the note is selected by interacting with the electronic device. Second user 1002 may select virtual representation 602 displayed in display 1100 if second user 1002 desires to interact with virtual note 108. Second user 1002 may select virtual representation 602 in any suitable manner”, and [0072]: “In step 906, the content input into the note is displayed”, and [0107]: “As described elsewhere herein, content 112 may include any of one or more types of content, such as text, graphics (e.g., images), video, audio, links, metadata, etc.”); and
providing the content of the digital element based at least in part on receiving the user selection indication (Par. [0073]: “In an embodiment, GUI window 1102 is opened after second user 1002 selects virtual representation 602 shown in FIG. 11. In the example of FIG. 12, content 112 includes a title 1208, "Best Burger in Town," an identifier 1210 for user 102 "Josh," a creation date 1212, "on Feb. 6, 2008," and a message body 1214, "Try the Garlic Fries, they're good!!!!" Second user 1002 can read this content provided by content 112, and any additional content that may be provided by content 112, including one or more images, videos, audio, links, etc.”).
Although Douris teaches providing a visual representation of the virtual note that indicates its presence at a real-world location, it is silent about: 1) providing a somatic representation of the virtual note in the manner described in the strike-through text above; and 2) processing updated geographical locations of the user device into a filtered location, and determining target rendering coordinates for the visual representation of the virtual note based on the filtered location.
Regarding 1, Latta teaches providing a somatic representation (e.g. a vibration) of a digital element that indicates a somatic presence of the digital element virtually located at a real world location, wherein the somatic representation is provided by performing a modification of the somatic representation based on a relative distance between the location of a user and the location of the digital element (Par. [0067] describes providing haptic feedback (e.g. a vibration of an electronic wristband around the wrist of the end user's hand that is within a particular distance of a virtual object) that increases with magnitude and/or frequency as the end user's hand gets closer to the virtual object).
In light of Latta’s teaching, it would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Douris by providing a somatic representation (e.g. a vibration of the user’s electronic device) of a virtual note that indicates a somatic presence of the virtual note attached to a real world location corresponding to the geographical location of the virtual note as a digitally obtainable object having the content that is different from the somatic representation, wherein the magnitude and/or frequency of the vibration is increased as the relative distance between the geographical location of the user’s electronic device and the real-world location where the virtual note is attached becomes smaller. The motivation would have been to provide a proximity feedback response (Latta, [0067]).
Regarding 2, Kokkas discloses a location-based augmented reality method in which a user's GNSS position is repeatedly updated and processed by a pre-processor implementing an extended Kalman filter, which combines each new GNSS observation with a prediction from the previous time instant to yield a "filtered position" (see pars. 53, 60, and 68). Kokkas further discloses that the resulting position, obtained from the filtered position, is used as the perspective center in the collinearity equations from which the image coordinates at which 3D map vertices and navigation instructions are drawn on the display are computed for each frame (see par. 75, including Eq. 23 and FIG. 7).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the Kalman-filtered location of Kokkas into the location-based overlay rendering of Douris, in order to reduce noise in the successive location updates, thereby reducing jitter in the rendered overlays, and improving the accuracy of registration between the virtual content and the real-world scene, as Kokkas teaches.
Regarding claim 2, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein providing the representation of the digital element includes providing the representation of the digital element using augmented reality (Douris, [0069]: “As shown in FIG. 11, display 1100 shows virtual representation 602 in a virtual space superimposed on a region similar to the region of image 302 shown in FIG. 3. Image 1102 is similar to image 302 because the camera of electronic device 1004 is positioned in a similar position, and pointed in a same direction as was the camera of electronic device 104 when capturing image 302”).
Regarding claim 3, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein providing the representation of the digital element includes generating an augmented reality representation of the digital element in a rendered view of at least a portion the geographical region of the digital element (Douris, [0069]: “In an embodiment, electronic device 1004 includes a camera that captures image 1102 displayed by display 1100… As shown in FIG. 11, display 1100 shows virtual representation 602 in a virtual space superimposed on a region similar to the region of image 302 shown in FIG. 3. Image 1102 is similar to image 302 because the camera of electronic device 1004 is positioned in a similar position, and pointed in a same direction as was the camera of electronic device 104 when capturing image 302”. In particular, virtual representation 602 is superimposed on a video image of a portion of the geographical region of entity 106 where virtual note 108 is placed).
Regarding claim 4, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein the content of the digital element is specified to be not obtainable when the distance difference between the geographical location of the user device and the geographical location of the digital element is greater than a threshold distance (Douris, [0068]: “In another embodiment, second user 1002 may interact with entity 106 by entering a region that is proximate to entity 106 (e.g., moving to within a predetermined distance of entity 106), or by entity 106 entering a region proximate to second user 1002 (if entity 106 is mobile). In any case, due to the interaction with entity 106 (assuming any access attributes associated with virtual note 108 allow for it), a display of electronic device 1004 may display virtual representation 602 of virtual note 108”. The cited text suggests that virtual representation 602 is not displayed if second user 1002 is outside the predetermined distance of virtual note 108. As a result, any content associated with virtual note 108 cannot be obtained).
Regarding claim 5, Douris in view of Latta and Kokkas discloses the method of claim 1, further comprising receiving the digital element from a server at least in part in response to sending an identifier of the geographical location of the user device to the server (Douris, [0072]: “In step 906, the content input into the note is displayed. As shown in FIG. 10, by interacting with entity 106, and selecting virtual representation 602, virtual note 108 is transmitted to electronic device 1004 (e.g., from a notes server, as described in further detail below) as indicated by arrow 1008 shown in FIG. 10. Virtual note 108 is received by electronic device 1004. Electronic device 1004 displays and otherwise enables second user 1002 to interact with virtual note 1008”. In particular, virtual note 108 is transmitted to electronic device 1004 in response to determining that electronic device 1004 is within a predetermined distance from entity 106 where virtual note 108 was placed. To determine whether electronic device 1004 is within the predetermined distance, the server compares its GPS coordinates to the coordinates of entity 106. The GPS coordinates of electronic device 1004 could be thought of as an identifier of its geographical location).
Regarding claim 6, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein the content of the digital element is specified to be only accessible by one or more specified users (Douris, [0053]: “In GUI window 502, user 102 may select one or more contacts 518 in contacts list portion 512, and then select (e.g., click on) one of family group selector 504, a friends group selector 506, a relatives group selector 508, and a create group selector 510 to provide the corresponding access level setting to the selected one or more contacts 518. For example, by selecting family selector 504, user 102 may assign the selected one or more contacts 518 "family" level access privileges for viewing, adding comments, and/or deleting virtual note 108”).
Regarding claim 7, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein the content of the digital element was provided by a first user for another user (In Douris, virtual note 108 (including any content associated with it) is created by a first user 102 for a second user 1002).
Regarding claim 8, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein the auditory content of the digital element is private to a single user (Douris, [0053]: “In GUI window 502, user 102 may select one or more contacts 518 in contacts list portion 512, and then select (e.g., click on) one of family group selector 504, a friends group selector 506, a relatives group selector 508, and a create group selector 510 to provide the corresponding access level setting to the selected one or more contacts 518. For example, by selecting family selector 504, user 102 may assign the selected one or more contacts 518 "family" level access privileges for viewing, adding comments, and/or deleting virtual note 108”. In particular, if user 102 does not select any contact, virtual note 108 (including any content associated with it) will not be shared with anybody).
Regarding claim 10, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein the content of the digital element is associated with a restriction based at least in part on a time when the content of the digital element is allowed to be accessed (Douris, [0133]: “As described above, user attributes may be used to filter virtual notes (e.g., by notes filter 2608) to enable access (e.g., viewing access, commenting access, deleting access, etc.) to virtual notes 108 for some users, while denying access to other users. For example, a variety of user attributes may be used to filter virtual notes, including any combination of time of day/week/year (temporal data), location (spatial data), relationships between users (social data), and topical data”).
Regarding claim 11, Douris in view of Latta and Kokkas discloses the method of claim 1, .
However, Douris discloses in [0133] that “user attributes may be used to filter virtual notes (e.g., by notes filter 2608) to enable access (e.g., viewing access, commenting access, deleting access, etc.) to virtual notes 108 for some users, while denying access to other users. For example, a variety of user attributes may be used to filter virtual notes, including any combination of time of day/week/year (temporal data), location (spatial data), relationships between users (social data), and topical data”, and in [0135] that “Spatial data 3202 may be any information associated with a location of a user and/or an electronic device associated with the user. For example, spatial data 3202 may include any passively-collected location data, such as cell tower data, GPRS data, global positioning service (GPS) data, WI-FI data, personal area network data, IP address data and data from other network access points, or actively-collected location data, such as location data entered into a device by a user”.
Therefore, it would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Douris by setting the user attributes of a virtual note such that if content of the virtual note includes any video, the video content can only be replayed or downloaded on the second user’s device when public Wi-Fi is available at the geographical region of the virtual note. The motivation would have been to prevent the second user’s device from consuming too much mobile data because video contents typically require more bandwidth than textual contents.
Regarding claim 12, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein the digital element is selectable within a predetermined distance from the geographical location of the digital element (Douris, [0068]. In particular, the fact that a virtual note can only be displayed on the second user’s device if the second user moves to within a predetermined distance from entity 106 where the virtual note is attached suggests the above limitation).
Regarding claim 13, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein providing the indication that the digital element is selectable includes providing a visual indication (See par. [0128] of Douris cited in the rejection of claim 1 above).
Regarding claim 14, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein performing the modification includes modifying an intensity of the somatic representation (See par. [0067] of Latta cited in the rejection of claim 1 above).
Regarding claim 17, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein providing the representation of the digital element includes using an image captured using a camera of the user device (Douris, [0069]: “In an embodiment, electronic device 1004 includes a camera that captures image 1102 displayed by display 1100. Display 1100 may have any suitable size, and be any type of display device, including those described above with respect to display 300 shown in FIG. 3. As shown in FIG. 11, display 1100 shows virtual representation 602 in a virtual space superimposed on a region similar to the region of image 302 shown in FIG. 3”).
Regarding claim 18, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein providing the indication that the digital element is selectable includes providing a notification (Douris, Fig. 30 and [0128]).
Claim 19 recites similar limitations as claim 1, but is directed to a system comprising one or more processors and a memory. Because Douris also discloses such a system (See Fig. 22), claim 19 can be rejected under the same rationale set forth in the rejection of claim 1.
Claim 20 recites similar limitations as claim 1, but is directed to a computer program product. Since Douris also discloses such a computer program product (See [0155]), claim 20 can be rejected under the same rationale set forth in the rejection of claim 1.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Douris, Latta and Kokkas as applied to claim 1 above, and further in view of Kawaki (Pub. No. US 2004/0098343).
Regarding claim 9, Douris in view of Latta and Kokkas discloses the method of claim 1, .
However, it should be noted that Douris discloses in [0051] and [0133] that the first user who creates a virtual note can associate attributes with it, wherein the attributes include access permissions, an expiration date, metadata associated with an area associated with the note, and alert attributes.
In the same field of content sharing, Kawaki teaches a technique for restricting accesses to shared content by employing use restriction rules which include the number of times a shared content is allowed to be accessed ([0049]: “It is to be noted that the use restriction rules to be registered in the use restriction information 19 include print permission to permit printing of image data displayed on the browser screen, save permission to permit data saving and screen copy permission to permit screen copying. The number of times the operation of printing, saving or the like is permitted, the number of times the image data can be reproduced and viewed, the period for viewing and so forth can also be registered as use restriction rules”).
In light of the above teaching of Kawaki, it would have been obvious to one skilled in the art before the effective filing date of the claimed invention to further modify Douris by including in the attributes associated with a virtual note a number of times the virtual note (and any audio content associated with it) can be accessed. The motivation would have been to prohibit illegitimate use of shared contents (Kawaki, [0012]).
Claim(s) 15-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Douris, Latta and Kokkas as applied to claim 1 above, and further in view of Salter et al. (Pub. No. US 2014/0375683).
Regarding claim 15, Douris in view of Latta and Kokkas discloses the method of claim 1, wherein performing the modification includes scaling or modifying a tactile representation based on the relative distance (See par. [0067] of Latta cited in the rejection of claim 1 above).
However, Douris in view of Latta and Kokkas does not disclose the above lined-through limitation.
In the same field of augmented reality, Salter teaches providing an audio/visual feedback (or any other suitable form, e.g. tactile/haptic feedback) for an object (which could be real or virtual) that indicates a presence of the object at a real world location, wherein the audio feedback is determined by performing an audio scaling or a modification based on a relative distance and an orientation between the geographical location of a user device and the real world location of the object (See pars. [0016], [0046] and [0052]-[0053]).
In light of the above teaching of Salter, it would have been obvious to one skilled in the art before the effective filing date of the claimed invention to further modify Douris by providing a somatic representation (e.g. haptic/tactile feedback) of the virtual note that indicates a somatic presence of the virtual note at a real world location located outside a field of view of the user device, wherein the somatic representation is determined by performing a scaling or a modification of the haptic/tactile feedback based on a relative distance and an orientation between the geographical location of the user device and the real world location of the virtual note. The motivation would have been to further alert the user, through haptic/tactile feedback, to information in the environment which is outside of the augmented reality field of view (Salter, [0016]).
Claim 16 could be rejected under the same rationale set forth in the rejection of claim 15 because Salter also describes using haptic feedback (See [0016] and [0053]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHONG X NGUYEN whose telephone number is (571)270-1591. The examiner can normally be reached Mon-Fri 8am - 5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHONG X NGUYEN/ Primary Patent Examiner, Art Unit 2617
