DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Applicant’s specification discloses paragraph [0026] In a second example, the text associated with the identified objects is usable to indirectly determine the objects' location by, e.g., describing the object with which the text is associated. A digital image, for instance, may capture a sign of a merchandise store (the object) with a banner containing the words “merchandise store.” Text indicating the name of the store may then be used as part of a lookup/search to locate the store with respect to a digital map. In this way, the location of the computing device used to capture the image is determined. Applicant’s specification paragraph [0027] In a third example, objects that do not include text are recognized using object recognition, and subsequently used as part of a search to determine a location. A computing device, for instance, may capture a live feed of digital images that include food items, drinks, bottles, and other items (i.e. objects) stacked on the shelves of a concession stand. Identification of the object may be used to determine the objects' location—on or near a concession stand. By consequence, the user's location is also marked as being in or near the concession stand with respect to a digital map. Applicant’s specification paragraph [0028] A computing device includes the location determination system may then leverage the determined location in a variety of ways. After determining the location of the object in relation to a digital map of a physical environment, for instance, the system generates augmented reality digital content that indicates the determined object location in relation to the digital image. Moreover, in one example, the augmented reality based digital content is configured as a map that guides the user from the location of the identified object to another location, e.g. the user's seat.
Applicant’s paragraph 0042 discloses “FIG. 4 depicts an example 400 of a physical environment 108 as a stadium. FIG. 5 depicts an example 500 of text included in a digital image that is directly indicative of a location. FIG. 6 depicts an example 600 of text included in a digital image that is indirectly indicative of a location, e.g., through identification of the object itself. FIG. 7 depicts an example 700 of objects included in a digital image and object recognition that does not include text that, by itself, is indicative of a location.”; paragraph 0045, To begin, at least one digital image 114 is received from a digital camera 112 by the camera platform manager module 116 as part of a live camera feed (block 802). The digital image 114, for instance may capture a physical environment 108, in which, the computing device 102 is disposed. From this digital image 114, a location determination system 120 of the camera platform manager module 116 determines a location 202; paragraph 0038, AR digital content 126, for instance, may describe a location of a seat, directions to the seat, a relation of that seat to other seats, directions to desired services available at the physical environment 106; paragraph 0054, An advertisement, for instance, may include informal language that does not identify the object, itself, but rather is usable to infer a location expressed by the text. The advertisement, for instance, may include directions to a corresponding store at the physical venue. From these directions, the natural language processing system 206 may determine a current location of the computing device 102 with respect to the physical environment 108; paragraph 0056, In a third example, the location is determined by the location determination system 120 using object recognition that is not based on text (block 812). A machine learning module 308, for instance, may include a plurality of models 310 that are trained as classifiers to determine a probability that a digital image includes a particular object. As shown in an example 700 of FIG. 7, for instance, the digital image 114 includes objects including drinks, a counter, popcorn, and a popcorn popper. The models 310 in this example generates object identifiers 306 based on this recognition. From the object identifiers, the location lookup module 312 determines a location 202. Continuing with the previous example, the object identifiers of the drinks, counter, popcorn, and so on may be used by the location lookup module 312 to infer that the computing device 102 is located proximal to a concession stand, and further that the concession stand sells popcorn. This knowledge may then be leveraged by the location lookup module 312 to locate concessions stands in relation to the digital map, and from this, determine the location 202. As a result, the location determination system 120 may leverage object recognition in a variety of different ways to determine a location
Examiner responds that Applicant has disclosed several exemplary embodiments, but fails to disclose the embodiment usable together as claimed. Applicant has failed to disclose how text in an object with direction, would identify an inferred location on a digital map of an environment. Applicant’s specification discloses the second example, the text associated with the identified objects is usable to indirectly determine the objects' location by, e.g., describing the object with which the text is associated. A digital image, for instance, may capture a sign of a merchandise store (the object) with a banner containing the words “merchandise store.” Text indicating the name of the store may then be used as part of a lookup/search to locate the store with respect to a digital map. In this way, the location of the computing device used to capture the image is determined.
Examiner responds Applicant discloses paragraph 0027, In a third example, objects that do not include text are recognized using object recognition, and subsequently used as part of a search to determine a location. A computing device, for instance, may capture a live feed of digital images that include food items, drinks, bottles, and other items (i.e. objects) stacked on the shelves of a concession stand. Identification of the object may be used to determine the objects' location—on or near a concession stand. By consequence, the user's location is also marked as being in or near the concession stand with respect to a digital map.
However, Applicant’s claim now recites “identifying, by the object recognition system, text included in the object that comprises directions to a location within the physical environment”. Therefore the claim is indefinite in that Applicant’s specification defined object that do not include text as recognized using object recognition.
Applicant’s specification paragraph 0511 discloses Natural language understanding as implemented by the natural language processing system 206 may then be used to understand what is being expressed by this language, and from this, infer a location of the computing device. An advertisement, for instance, may include informal language that does not identify the object, itself, but rather is usable to infer a location expressed by the text. The advertisement, for instance, may include directions to a corresponding store at the physical venue. From these directions, the natural language processing system 206 may determine a current location of the computing device 102 with respect to the physical environment 108.
[0052 In this way, the location determination system 120 may leverage text that directly indicates a location (e.g., a section, seat number), indirectly indicates a location through identification of an object (e.g., a name of a store), and even more indirectly through text that does not identify the object nor the location. Non-textual techniques object recognition techniques may also be employed by the location determination system 120 as described in the following example. [00531 In a third example, the location is determined by the location determination system 120 using object recognition that is not based on text (block 812). A machine learning module 308, for instance, may include a plurality of models 310 that are trained as classifiers to determine a probability that a digital image includes a particular object. As shown in an example 700 of FIG. 7, for instance, the digital image 114 includes objects including drinks, a counter, popcorn, and a popcorn popper. The models 310 in this example generates object identifiers 306 based on this recognition. From the object identifiers, the location lookup module 312 determines a location 202. Continuing with the previous example, the object identifiers of the drinks, counter, popcorn, and so on may be used by the location lookup module 312 to infer that the computing device 102 is located proximal to a concession stand, and further that the concession stand sells popcorn.
Examiner therefore determines the claim language as recited in claims 1, 11 and 16 as indefinite in that Applicant has failed to define how “an object recognition system, using a classifier of the object recognition system,” where the classifier as described above by Applicant’s specification does not include text, but as example includes recognition of popcorn, drinks and infer that the device is proximal to a concession stand; “identifying text in the object, by an object recognition system, text included in the object”, such as an advertisement as exampled by Applicant’s specification, “generating, by natural language processing system an inferred location”, such as a captured image of a ticket to a venue, as disclosed by Applicant’s specification.
Claims that are noted above as being rejected but not specifically addressed are rejected based upon dependency upon rejected independent claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-12, 14-18 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fire et al., U.S. Patent Number 10,606,824 B1, in view of Smith et al., U.S. Patent Publication Number 2020/0327378 A1, further in view of Xu et al., U.S. Patent Publication Number 2015/0233715.
Regarding claim 1, Fire discloses a method for location determination, the method comprising: receiving, by a computing device, at least one digital image of a physical environment (col. 7, lines 25-26, receive images of objects at geographic locations, which Examiner interprets as a physical environment); identifying, by an object recognition system at the computing device, an object in the at least one digital image using a classifier of the object recognition system (col. 7, lines 26-30, a classifier component can analyze the image data to determine, for example, visual features (e.g., feature points, feature descriptors, etc.); col. 7, lines 51-56, the classifier component can be part of an object recognition system configured to recognize point and/or objects of interest); identifying, by the object recognition system, text included in the object that comprises directions to a location within the physical environment (col. 2, lines 58-65, determine visual features; visual features can include, for example, text or images; represented in the image data; col. 13, lines 44-47, features (e.g., shape, size, color and text)); generating, by a text recognition process at the computing device (col. 13, lines 56-59, various other techniques (e.g., OCR and other text recognition processes) can be used), an inferred location of the object within the physical environment based on text recognition process understanding of the directions, wherein the inferred location is different from the location (col. 16, lines 40-50, the device can utilize outputs from at least one of the image capture elements to assist in determining the location and/or orientation of a user and in recognizing nearby persons, objects or locations; for example, if the user is hold the device, the capture image can be analyzed (e.g., using mapping information about a particular area) to determine the approximate location), Examiner interprets determining nearby location as an inferred location); identifying, by the computing device, the location of the object in relation to mapping information of the physical environment based on the inferred location (col. 12, lines 59-67, image data can include at least one visual feature; tracking information can be matched to stored information; stored information can correspond to a one or more items used to determine an item matching the visual feature; col. 16, lines 48-50, the captured image information may also be analyzed to recognize nearby persons, object or locations (e.g., by matching parameters or elements from the mapping information); col. 15, lines 59-60, can be extended to other services such as mapping services, navigation services); and displaying, by the computing device, a current location of the computing device on the image data based on the location of the object (col. 13, lines 1-13, the item matched to the visual feature can be retrieved and provided for display with the image data; augment the image data by overlaying the content, wherein the overlay element is selected from one of a three-dimensional (3D) structure).
However, it is noted that Fire discloses matching parameters or elements from the mapping information, but fails to specifically disclose
natural language processing system at the computing device, an inferred location of the object within the physical environment based on a natural language understanding of the directions; identifying, by the computing device, the location of the object in relation to a digital map of the physical environment based on the inferred location; and displaying, by the computing device, a current location of the computing device on the digital map based on the location of the object.
Smith discloses identifying an object (paragraph 0022, identifying objects or things); identifying, by the object recognition system, text included in the object (paragraph 0027, objects may include or reference items of content, such as text); generating, by a natural language processing system at the computing device, an inferred location of the object within the physical environment based on a natural language understanding of the directions, wherein the inferred location is different from the location (paragraph 0097, location of a user may be inferred from information associated with a Global Positioning System or any other position or location aware device; may be inferred directly from location information input by a user; or otherwise acquired by a computer-based system; paragraph 0142, natural language processing techniques that are applied to text strings such as sentences with computer implemented object); identifying, by the computing device, the location of the object in relation to a digital map of the physical environment based on the inferred location (paragraph 0097, inferences may be made in conjunction with geographic contextual information or system, such as through interaction with digital maps).
However, it is noted that Smith fails to specifically disclose text that comprises directions to a location within the physical environment.
Xu discloses receiving, by a computing device, at least one digital image of a physical environment (paragraph 0009, a device typically displays the live view of the physical, real-world environment on a screen; paragraph 0023, receive ticket information; paragraph 0034, may scan a physical ticket to provide the ticket information); identifying, by the object recognition system, text included in the object that comprises directions to a location within the physical environment (paragraph 0034, ticket information includes, but is not limited to, the event, seat information (e.g., section, row and number), etc.,; paragraph 0037, navigates or guides the user to his or her seat); generating, by a natural language processing system at the computing device, an inferred location of the object within the physical environment based on a natural language understanding of the directions, wherein the inferred location is different from the location; identifying, by the computing device, the location of the object in relation to a digital map of the physical environment based on the inferred location (paragraph 0037, may use the camera of user device such as an iPhone or iPad to receive directions); and displaying, by the computing device, a current location of the computing device on the digital map based on the location of the object (paragraph 0038, may show arrows or a map to augmented the user’s view of the venue; a two-dimensional indoor map of the venue augments the user’s view of the venue on the user device, along with arrows that point in the direction the user should be walking).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to include in the text as disclosed by Fire, using natural language processing as disclosed by Smith, to provide other techniques (e.g., OCR and other text recognition processes) to process text identified in an image, as disclosed by Fire. It further would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include in the display as disclosed by Fire, displaying the location in relation to a digital map as disclosed by Smith, and displaying a current location on the digital map, as disclosed by Xu, to allow a user to recognize nearby persons, object or locations (e.g., by matching parameters or elements from the mapping information), and provide information such as position, direction, motion or orientation where the approach is extended to services such as mapping service, navigation services, as disclosed by Fire.
Regarding claim 2, Fire discloses further comprising: collecting, by the computing device, data pertaining to the object via a search (col. 3, lines 16-18, collecting images of points and/or object at a geographic location; col. 11, lines 24-26, database that the online directory server uses to respond to user request and search queries); generating, by the computing device, augmented reality digital content indicating the data pertaining to the object (col. 12, lines 45-67, image data or point or object can be received at the augmented reality platform; determine an item matching to the visual feature); and displaying, by the computing device, the augmented reality digital content (col. 13, lines 1-3, the item matched to the visual feature can be retrieved and provided for display with the image data on the client computing device; the content can be rendered in an overlay element that overlays the image data displayed on the computing device).
Smith discloses a digital map (paragraph 0097, inferences may be made in conjunction with geographic contextual information or system, such as through interaction with digital maps).
Regarding claim 3, Fire discloses wherein the augmented reality digital content is rendered as part of a live camera feed of the physical environment
(col. 13, lines 3-5, content can be rendered in an overlay element that overlays the image data displayed; col. 13, line 33, capture a live view).
Regarding claim 4, Fire discloses wherein the augmented reality digital content comprises digital content (col. 12, lines 31-35, augmented reality platform can provide a web service allowing user to search and discover links and other content (e.g., reviews, menus, video, chat walls, contact information, URLs) that are tied to unique visual features at a point of interest in the real world; all of which Examiner interprets as digital marketing content).
Smith discloses digital marketing content (paragraph 0227, identifying objects in videos that can, in accordance with some embodiments, be applied for video-based user-responsive product promotion purposes; promoted is received from the studio, or from marketers of the products ).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include digital marketing content as disclosed by Smith to provide user-responsive product promotional purposes.
Regarding claim 5, Fire discloses wherein the physical environment is a venue, an event at the venue (col. 14, 5-6, restaurant is decorated for a special event or seasonal offerings).
However, it is noted that Fire and Smith fail to disclose the augmented reality digital content comprises ticket information for an event at the venue.
Xu 2015/0233715 wherein the physical environment is a venue, an event at the venue (paragraph 0009, assisting a user at a venue); and the augmented reality digital content comprises ticket information for an event at the venue (paragraph 0011, in an augmented reality environment, relevant information regarding a location, event or venue can be rendered or presented to the user; paragraph 0034, provides ticket information associated with an event; ticket information include, but is not limited to, the event, the venue, seat information, etc.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include in the event as disclosed by Fire, ticket information for the event at a venue, as disclosed by Fire, to provide relevant information regarding the venue to be presented to a user, such as where the user is to be seated at the event.
Regarding claim 6, Fire discloses wherein the classifier is trained via machine learning (col. 8, lines 330-31, machine learning algorithm can be applied to classify the object).
Regarding claim 7, Fire discloses wherein the image data comprises a 3D model of the physical environment (col. 13, lines 1-13, the item matched to the visual feature can be retrieved and provided for display with the image data; augment the image data by overlaying the content, wherein the overlay element is selected from one of a three-dimensional (3D) structure).
Smith discloses a digital map ((paragraph 0097, inferences may be made in conjunction with geographic contextual information or system, such as through interaction with digital maps).
Xu discloses wherein the directions are to a destination within the physical environment, and wherein the natural language processing system is configured to understand what is being expressed by the directions to generate the inferred location based on the destination (paragraph 0039, overlays the time left and distance to the seat on the user 102's view. For example, the distance left to get to the seat may be displayed as 150 meters, and the time to get to the seat may be displayed as 2 minutes. As the user 102 gets closer to his or her seat, the time and distance decrease. In various embodiments, the service provider determines the walking speed of the user 102, and employs this information to provide an estimate of how much longer and how much farther it will take for the user 102 to get to his or her seat).
Regarding claim 8, Fire discloses wherein the receiving the at least one digital image is responsive to detecting, by the computing device, a lack of signal for a wireless position determining functionality to determine the current location of the computing device (col. 1, lines 35-40, where the search result are not available or lacking; user may not be able to locate potential places or business or other locations).
Regarding claim 9, Fire discloses wherein the at least one digital image is received using a digital camera of the computing device as part of a live camera feed of the physical environment (col. 13, lines 33-34, capture a live view of at least a portion of ABC Restaurant).
Regarding claim 10, Fire discloses further comprising: verifying the current location of the computing device based on a previously determined location of a previously identified object and the physical environment, wherein the verifying is based on continued monitoring of one or more locations (col. 13, lines 60-67, receive the image data and attempt to quickly verify the restaurant as may include information associated with the restaurant; visual features can be compared information stored in a database or other storage device; a change in the visual representation of the restaurant can be determined; col. 14, lines 1-11, change might be due to, for example, a change in the restaurant operating at that location; a change in the visual appearance; restaurant is decorated for a special event or seasonal offerings; an update to restaurant hours; update to other information, all of which Examiner interprets as continued monitoring).
Smith discloses a digital map ((paragraph 0097, inferences may be made in conjunction with geographic contextual information or system, such as through interaction with digital maps).
Regarding claim 11, it is rejected based upon similar rational as above claim 1. Fire further discloses a computing device, comprising: a processing system; and a computer-readable storage medium storing instructions that, responsive to execution by the processing system (figure 6).
Regarding claim 12, Fire discloses wherein the receiving is performed responsive to detecting a lack of signal for a wireless position determining functionality of the computing device (col. 1, lines 35-40, where the search result are not available or lacking; user may not be able to locate potential places or business or other locations).
Regarding claim 14, Fire discloses wherein the computer-readable storage medium stores further instructions that, responsive to execution by the processing system, cause the processing system to perform operations comprising: collecting, by the computing device, data pertaining to the object via a search (col. 3, line 19, collecting images; col. 11, lines 51-56, a user can perform a search); generating, by the computing device, augmented reality digital content indicating the data pertaining to the object (col. 12, lines 31-33, augmented reality platform can provide a web service allowing users to search and discover links and other content that are tied to unique visual features at a point of interest in the real world); and displaying, by the computing device, the augmented reality digital content proximal to the location of the object (col. 13, lines 3-5, content can be rendered in an overlay element that overlays the image data displayed on the computing device).
Smith discloses (paragraph 0097, inferences may be made in conjunction with geographic contextual information or system, such as through interaction with digital maps).
Regarding claim 15, Fire discloses generating information describing characteristics of the physical environment by processing the at least one digital image using machine learning (col. 8, lines 30-31, a machine learning algorithm can be applied to classify the object; col. 10, lines 35-40, identifying information corresponding to the database object and/or point of interest can be provided to the object service).
It is noted that Fire claims recommendations (claim 3).
Smith specifically discloses recommendations (paragraph 0023, adaptive recommendations; paragraph 0039, recommended content or activities may include information generated automatically by a processor-based system or device, such as a process control device. A recommendation may comprise a spatial or temporal sequence of objects. Adaptive recommendations 250 may be in the context of a currently conducted activity of system 100, a current position while navigating structural aspect 210).
It is noted that Fire and Smith fail to disclose outputting a recommendation based on the characteristics.
Xu discloses outputting a recommendation based on the characteristics (paragraph 0040, route may provide suggestions for restrooms and/or concession stands that are less congested along the way to the seat; paragraph 0043, information relevant to the user’s personal choices as the user moves the user device within the venue).
It would have been obvious to one of ordinary skill in the art to combine the machine learning as disclosed by Fire for objects in a captured location, with the recommendation of object as disclosed by Smith, providing the suggestions as disclosed by Xu, to provide a user with suggestions along a navigated route using machine learning of classification of objects to indicate which objects along the route should be suggested.
Regarding claims 16 -18, they are rejected based upon similar rational to claims 1, 3 and 8 respectively. Fire discloses a method for location determination (col. 2, lines 7-14).
Regarding claim 20, Fire discloses wherein the machine learning model is trained using training digital images obtained from a commerce service provider system that are tagged by sellers using the commerce service provider system (col. 6, lines 49-56, address/locale-correlated image and other information may be published in a content aggregation system; provides a web service allowing user and other entities to obtain images and other content (e.g., reviews, menus, business information, etc.) that are associated with visual features at a point of interest in the real world; col. 14, lines 20-26, content aggregation system can include a classifier service and an update service, which Examiner interprets as a commerce service provider, in that the provider, of content can provide updates or changes, col. 14, lines 1-12).
Claim(s) 13 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fire, Smith and Xu as applied to claims 11 and 16 above, and further in view of U.S. Patent Publication Number Narasimhan 2018/0131906.
Regarding claims 13 and 19, it is noted that Fire, Smith and Xu disclose identifying, by the computing device, a current location of the computing device based on the location of the object; displaying, by the computing device, the current location of the computing device on the digital map (Fire 10,606,824, col. 2, lines 50-55, information can include, e.g., GPS coordinates, navigation and orientation information derived from a compass, distance information of the computing device to the object and/or point of interest of the computing device; Smith paragraph 0097, inferences may be made in conjunction with geographic contextual information or system, such as through interaction with digital maps; Xu a two-dimensional indoor map of the venue augments the user’s view of the venue on the user device, along with arrows that point in the direction the user should be walking).
However, Fire, Smith and Xu fail to disclose receiving data describing where access is permitted in the physical environment; and generating navigation from the current location to a desired location on the digital map based on the data describing where access is permitted.
Narasimhan 2018/0131906 discloses receiving data describing where access is permitted in the physical environment; and generating navigation from the current location to a desired location on the digital map based on the data describing where access is permitted (paragraph 0014, to enable the exchange of service-based content via the wireless communication network, the system may comprise one or more wireless access points (WAPs) position through the venue; paragraph 0021, information pertaining to the venue, such as, for example, restrooms and concession stand locations near a particular seating area; paragraph 0025 required fee at a venue location; identifying information may be input by an attendant at a venue kiosk; paragraph 0025, WAP’s may be located such that the services are only accessible within one or more designated areas of the venue; paragraph 0051, provide requested non-video content in the form of navigable web pages).
It would have been obvious to one of ordinary skill in the art to include navigation to a wireless access kiosk at a venue as disclosed by Narasimhan, in the captured image system as disclosed by Fire, to provide navigation to items of interest, such as wireless access whereas disclosed by Fire col. 1, lines 35-40, where the search result are not available or lacking; user may not be able to locate potential places or business or other locations.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Chakraborty et al., U.S. Patent Publication Number 2016/0247023 A1
Chakraborty discloses receiving, by a computing device, at least one digital image of a physical environment (paragraph 0030, receives user input, which may include any forms of input; (e.g., text, audio, video, image); user input includes imagery captured “live” by a wearable computing system, any type of mobile device, or other computing device; paragraph 0032, receive user input in the form of “live” (e.g., streaming) video); identifying, by an object recognition system at the computing device, an object in the at least one digital image using a classifier of the object recognition system (paragraph 0032, processes the video image; intelligent assistance (e.g., suggestions, notifications, etc.) prepared by the intelligent assistant based on the classifications generated by the entity interaction system; paragraph 0034, entity interaction recognition system is embodied as an interactions knowledge base, a perspective rectification module, and a classification module; paragraph 0035, interactions knowledge base provides data representations of knowledge in a variety of areas that are relevant to the analysis of entity interactions in images; contain data that indicate relationships between information that can be extracted from images; and a likely semantic meaning of such extracted information; provides rules, mappings, probability distributions, statistical likelihoods, or other relationship indicators that allow the entity interaction recognition system to interpret information that is extracted from an image; paragraph 0036, portions of the data in the interactions knowledge base may be structured; other portions may be unstructured (e.g., natural language or free-form text); interactions knowledge base may contain or reference data, arguments, parameters, and/or machine-executable algorithms that can be applied to the analysis and classification of images as described herein)
paragraph 0039, object models 214 provide computer-accessible representations of knowledge and assumptions about the typical characteristics or features of known entities other than people (such as various objects) that may be detected in an image, and corresponding semantic information that describes such entities; paragraph 0051, entity-object interaction models provide computer-accessible representations of knowledge and assumptions about the typical characteristics or features of entity interactions (e.g., human or non-human entities with other non-human entities (e.g., surfaces, objects, etc.), which may be detected in an image, and corresponding semantic information; define relationships between particular spatial configurations; entity-object arrangements;
;paragraph 0041, proxemics-based attributes 218 specify relationships between categories of measurable features that can be detected in images or estimated therefrom, and the corresponding proxemics-based interpretations of such features; paragraph 0053, maps people/face/entity locations from the 2D scene space to the 3D space; paragraph 0030, output may include graphical or textual elements that are inserted into or overlay at least a portion of the viewing area of the user’s viewing device (as in an augmented reality system).
Amacker et al., U.S. Patent Number 9,792,368 B1
Amacker discloses col. 3, line 63, receives the captured image information; col. 2, line 58-59, at least one image recognition process can be used to match the captured image to existing and/or stored map; col. 7, lines 22-30, map information can be analyzed to attempt to locate interesting or distinct features that can be matched with feature found in images taken by users; features need not be highways or intersections, for example, buy can include features such as map legends and corners of letters on the map.
Bergmann et al., U.S. Patent Publication Number 2008/0056535 A1
Bergmann discloses paragraph 0008, retrieving image data; paragraph 0009, recognizing an object in an image; paragraph 0034, navigation system may also include information on the shape and/or orientation of specific objects such as, for example, the orientation of individual road segments; traffic signs, the digital map information may include only a qualifier specifying the type of object, such as for example, for a stop sign; paragraph 0035, predict where the camera is located on the map; it is possible to retrieve only that portion of the digital map required for recognizing the object in the image; paragraph 0028, verify correct object recognition in the image data.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Motilewa Good-Johnson whose telephone number is (571)272-7658. The examiner can normally be reached Monday - Friday 6am-2:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MOTILEWA . GOOD JOHNSON
Primary Examiner
Art Unit 2616
/MOTILEWA GOOD-JOHNSON/Primary Examiner, Art Unit 2619
Prosecution Insights