DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action s in response to the Amendment filed on 4/6/2026.
Claims 1-20 are pending. Claims 1, 3, 9, 10 have been amended. Claims 18 has been cancelled.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/6/2026 has been entered.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-10 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-10 of app 17/689,931. Although the claims at issue are not identical, they are not patentably distinct from each other because they both claim the same subject matters and limitations as explained below.
Claim 1 is determined to be obvious in light of claim 1 of application 17/689,931 based on reasons below for having similar limitations.
Instant Application Claim 1 (17/689,929)
Reference Application Claim 1 (17/689,931)
A method of interacting from a stadium seat object comprising: enabling capture by a portable camera of an image of a real-world stadium seat object having optically-readable indicia disposed thereon; using at least one processor, recognizing the real-world stadium seat object based on the optically-readable indicia from the captured image without decoding any optical code or optical marker; associating the recognized real-world stadium seat object with an interactive media record in response to the recognizing; presenting an interactive media item for visual presentation, including superimposing the interactive media item onto a real time display of an image captured by the portable camera to appear as if the superimposed interactive media item were part of a real world scene captured by the portable camera even though the superimposed interactive media item is virtual; enabling user interaction with the superimposed interactive media item; and triggering purchasing and/or shopping from the stadium seat object in response to received user interaction with the superimposed interactive media item.
A method of presenting ticket information to a user comprising: enabling capture by a portable camera of an image of a ticket object having at least one distinctive image pattern that is not a QR code or bar code disposed thereon in an area of recognition; recognizing the at least one distinctive image pattern from the captured image of the ticket object in real time without requiring recognition outside the area of recognition; associating, in real time, the recognized ticket object with a record in response to the recognizing; selecting, in real time, an interactive media item in response to the associating; and superimposing, in real time, the selected interactive media item onto a display of the captured image or an image derived therefrom.
Although the claims at issue are not identical, they are not patentably distinct from each other. Both claims are directed to a method of capturing an image of a physical object bearing indicia, recognizing those indicia in real time from the captured image, associating the recognized object with a record, and superimposing an interactive media item onto a display of the captured image in response to the recognition and association. The instant application's claim 1 recites "stadium seat object" with "optically-readable indicia" and "triggering purchasing and/or shopping," while reference claim 1 recites "ticket object" with "at least one distinctive image pattern that is not a QR code or bar code." A stadium seat object is a species of physical real-world objects bearing readable indicia, and the broader concept of optically-readable indicia on a physical object is equivalent in scope to a distinctive image pattern on a ticket object. The "triggering purchasing and/or shopping" limitation of the instant claim 1 is an obvious extension of the selection and superimposition of the interactive media item taught in reference claim 1, as it would have been obvious to one of ordinary skill in the art to implement a purchasing/shopping trigger as a natural and predictable use of an interactive media item superimposed in response to recognizing a physical object bearing indicia. Therefore, claim 1 of 17/689,929 is not patentably distinct from claim 1 of 17/689,931.
Claim 2 is determined to be obvious in light of claim 2 of application 17/689,931 based on reasons below for having similar limitations.
Instant Application Claim 2 (17/689,929)
Reference Application Claim 2 (17/689,931)
The method of claim 1 wherein the superimposing comprises using at least one of augmented reality, mixed reality and virtual reality.
The method of claim 1 wherein the superimposing comprises using at least one of augmented reality, mixed reality and virtual reality.
Although the claims at issue are not identical due to their different respective base claims, they are not patentably distinct from each other. Claim 2 of 17/689,929 and claim 2 of 17/689,931 recite the identical limitation — that the superimposing comprises using at least one of augmented reality, mixed reality and virtual reality. The limitations are identical in scope and content. Therefore, claim 2 of 17/689,929 is not patentably distinct from claim 2 of 17/689,931.
Claim 3 is determined to be obvious in light of claim 3 of application 17/689,931 based on reasons below for having similar limitations.
Instant Application Claim 3 (17/689,929)
Reference Application Claim 3 (17/689,931)
The method of claim 1 wherein recognizing comprises recognizing a three-dimensional real-world stadium seat object with a seat number printed on it and the method further includes determining location of the three-dimensional real-world stadium seat object within a stadium, concert hall, theater or other venue in response to the recognizing, and initiating delivery of an ordered or purchased item to the determined location of the three-dimensional real-world stadium seat object in response to the triggered purchasing and/or shopping.
The method of claim 1 wherein the recognition comprises recognizing a two-dimensional or three-dimensional ticket object with at least one distinctive image pattern printed on it.
Although the claims at issue are not identical, they are not patentably distinct from each other. Both claims are directed to recognizing a physical object — either a stadium seat or a ticket — that has printed indicia on it, in the context of the same underlying AR recognition method of claim 1 of each respective application. Claim 3 of the instant application additionally recites determining the location of the stadium seat object and initiating delivery; however, these limitations are an obvious implementation of the real-time recognition and association steps already established in claim 1 of 17/689,929, and would have been obvious to one of ordinary skill in the art to include as a natural and predictable extension of recognizing a specific seat object within a venue context. Therefore, claim 3 of 17/689,929 is not patentably distinct from claim 3 of 17/689,931.
Claim 4 is determined to be obvious in light of claim 4 of application 17/689,931 based on reasons below for having similar limitations.
Instant Application Claim 4 (17/689,929)
Reference Application Claim 4 (17/689,931)
The method of claim 1 wherein the stadium seat object has optical indicia printed thereon, and the recognizing comprises recognizing at least some of the printed optical indicia.
The method of claim 1 wherein the ticket object has the at least one distinctive image pattern printed thereon, and the recognizing comprises recognizing the printed at least one distinctive image pattern.
Although the claims at issue are not identical, they are not patentably distinct from each other. Both claims recite that the physical object — a stadium seat object or a ticket object — has printed indicia thereon, and that the recognizing step comprises recognizing at least some of the printed indicia. The instant claim 4 uses the term "optical indicia" while the reference claim 4 uses "at least one distinctive image pattern"; however, both terms refer to printed visual marks on a physical object that are recognized from a captured image, and the scope of both limitations is substantively equivalent. Therefore, claim 4 of 17/689,929 is not patentably distinct from claim 4 of 17/689,931.
Claim 5 is determined to be obvious in light of claim 5 of application 17/689,931 based on reasons below for having similar limitations.
Instant Application Claim 5 (17/689,929)
Reference Application Claim 5 (17/689,931)
The method of claim 4 wherein the recognizing includes recognizing, using at least one processor in a network, characters printed on the stadium seat object and permitting a remote wearable and/or holdable device bearing the portable camera to access, over an electronic wireless network, a database entry or record stored in the cloud and matched to the recognized characters printed on the stadium seat object.
The method of claim 4 wherein the recognizing further includes recognizing characters printed on the ticket object.
Although the claims at issue are not identical, they are not patentably distinct from each other. Both claims are directed to recognizing characters printed on the physical object. The instant claim 5 additionally recites cloud database access over a wireless network via a wearable or holdable device; however, accessing a cloud-based database over a wireless network in response to character recognition is an obvious and predictable implementation of the character recognition step recited in reference claim 5, as accessing network resources in response to recognized data was well-established in the art at the time of the invention. Therefore, claim 5 of 17/689,929 is not patentably distinct from claim 5 of 17/689,931.
Claim 6 is determined to be obvious in light of claim 6 of application 17/689,931 based on reasons below for having similar limitations.
Instant Application Claim 6 (17/689,929)
Reference Application Claim 6 (17/689,931)
The method of claim 5 wherein the optically-readable indicia on the stadium seat object comprises a patch of printed material attached to the stadium seat object.
The method of claim 5 wherein the ticket object comprises a patch of printed material attached to an associated item.
Although the claims at issue are not identical, they are not patentably distinct from each other. Both claims recite that the physical object — a stadium seat object or a ticket object — comprises a patch of printed material attached thereto. The instant claim 6 specifies the patch is attached to the "stadium seat object" while reference claim 6 specifies attachment to an "associated item." A stadium seat object is a species of associated item to which a patch of printed material may be attached, and the two limitations are functionally and structurally equivalent. Therefore, claim 6 of 17/689,929 is not patentably distinct from claim 6 of 17/689,931.
Claim 7 is determined to be obvious in light of claim 7 of application 17/689,931 based on reasons below for having similar limitations.
Instant Application Claim 7 (17/689,929)
Reference Application Claim 7 (17/689,931)
The method of claim 6 wherein the interactive media item comprises a digital overlay that leads to specific marketing action relating to the recognized stadium seat object selected from the group consisting of providing specific marketing information; a video, tutorial, or displayable marketing content relating to the recognized stadium seat object.
The method of claim 6 wherein the selected interactive media item comprises a digital overlay that leads to specific action selected from the group consisting of providing specific information; a video, tutorial, or any kind of displayable content.
Although the claims at issue are not identical, they are not patentably distinct from each other. Both claims recite that the interactive media item comprises a digital overlay that leads to a specific action selected from the group consisting of specific information, a video, tutorial, or displayable content. The instant claim 7 qualifies the action and content as "marketing" in nature; however, providing marketing information, videos, tutorials, and other displayable marketing content relating to the recognized object is a species of the broader "specific action" and "specific information" recited in reference claim 7. Marketing content and marketing actions are obvious and predictable applications of the displayable content and specific actions expressly taught in the reference claim. Therefore, claim 7 of 17/689,929 is not patentably distinct from claim 7 of 17/689,931.
Claim 8 is determined to be obvious in light of claim 8 of application 17/689,931 based on reasons below for having similar limitations.
Instant Application Claim 8 (17/689,929)
Reference Application Claim 8 (17/689,931)
The method of claim 1 wherein the superimposing is performed on a display provided by smart glasses.
The method of claim 1 wherein the superimposing is performed on a handheld display device, a user's retina or smart glasses.
Although the claims at issue are not identical, they are not patentably distinct from each other. Reference claim 8 of 17/689,931 expressly recites smart glasses as one of the display options on which superimposing is performed. The instant claim 8 recites only "a display provided by smart glasses," which is a narrower subset of the display options recited in the reference claim. Since the reference claim 8 of 17/689,931 expressly discloses smart glasses as a display device for the superimposing step, the instant claim 8 is anticipated by — and at minimum obvious over — the reference claim. Therefore, claim 8 of 17/689,929 is not patentably distinct from claim 8 of 17/689,931.
Claim 9 is determined to be obvious in light of claim 9 of application 17/689,931 based on reasons below for having similar limitations.
Instant Application Claim 9 (17/689,929)
Reference Application Claim 9 (17/689,931)
The method of claim 1 wherein the interactive media item comprises a call button to call a service to the stadium seat object.
The method of claim 1 wherein the selected interactive media item comprises a call button.
Although the claims at issue are not identical, they are not patentably distinct from each other. Both claims recite that the interactive media item comprises a call button. The instant claim 9 further specifies that the call button is to "call a service to the stadium seat object"; however, this qualifier identifies only the intended purpose of the call button in the context of the stadium seat recognition method already established in claim 1 of 17/689,929. It would have been obvious to one of ordinary skill in the art that a call button in an AR system triggered by recognition of a stadium seat object would be used to call a service relating to that stadium seat object. Therefore, claim 9 of 17/689,929 is not patentably distinct from claim 9 of 17/689,931.
Claim 10 is determined to be obvious in light of claim 10 of application 17/689,931 based on reasons below for having similar limitations.
Instant Application Claim 10 (17/689,929)
Reference Application Claim 10 (17/689,931)
The method of claim 1 further including displaying any or all of the following action buttons in any combination or subcombination: Price Tag, Photo Gallery of purchasable products or services, Videos of purchasable products or services, Description of purchasable products or services, Call, Shop link to buy merchandise or tickets, Map of a stadium, Discount Codes, Order food, and/or Booking opportunities.
The method of claim 1 further including displaying any or all of the following action buttons in any combination or subcombination: Price Tag, Photo Gallery, Videos, Description, Call, Mail, Shop link merchandise, Explanation, Intro, Social Media links, Map, Discount Codes, Reviews, Directions, Booking opportunities, and/or Seat information.
Although the claims at issue are not identical, they are not patentably distinct from each other. Both claims are directed to displaying a menu of action buttons in any combination or subcombination in the AR interface triggered by recognition of a physical object. The instant claim 10 recites a subset of the action buttons found in reference claim 10 (Price Tag, Photo Gallery, Videos, Description, Call, Discount Codes, Booking opportunities), with the addition of "Map of a stadium" and "Order food." The "Map of a stadium" is a species of the "Map" and "Directions" buttons expressly recited in reference claim 10, and "Order food" is an obvious species of the "Shop link merchandise" and "Booking opportunities" buttons recited in the reference claim, as ordering food is a form of merchandise shopping and service booking. Therefore, claim 10 of 17/689,929 is not patentably distinct from claim 10 of 17/689,931.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4-8, 10, 11, 14-18, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wan (US 20140111542 A1), in view of Nurmi (US 20120221241 A1), further in view of Mayerle et al (US 20140100997 A1, hereinafter " Mayerle").
Regarding Claim 1, Wan teaches a method of interacting from a [[ stadium seat ]] object (Wan, Paragraph [0065], "These advantages improve the user experience and enable further information to be retrieved relating to the user's present visual environment"; [0131], “a smartphone, it can be used to interact with the mobile application 30”) comprising:
enabling capture by a portable camera of an image of a real-world [[ stadium seat ]] object having optically-readable indicia disposed thereon (Wan, Paragraph [0013], "a display module for displaying the live video feed <read on image of a real-world object> on a screen of the mobile device"; [0077], "The text markers 80 in the live video feed 49 for detection by the OCR engine 32 may be found on printed or displayed matter 70 <read on optically-readable indicia disposed thereon>, for example, outdoor advertising, shop signs, advertising in printed media, or television or dynamic advertising light boxes");
using at least one processor, recognizing the real-world [[ stadium seat ]] object based on the optically-readable indicia from the captured image (Wan, Paragraph [0076], "The OCR engine 32 detects the presence of text in a live video feed 49 captured by the built-in device video camera 21 in real-time, and converts the detected text 41 into machine-encoded text in real-time"; [0100], "When the OCR engine 32 has detected text 41 in the live video feed 49, it converts 183 it into machine-encoded text and a query 184 on the database 35, 51 is performed");
without decoding any optical code or optical marker (Wan, Paragraph [0007], "However, this process is cumbersome as it requires businesses to work closely with the advertising agency in order to place the QR code at a specific position of the outdoor advertisement. This wastes valuable advertising space, and the QR code only serves a single purpose to small percentage of passer bys and therefore has no significance to the majority of passer bys"; [0086], "they do not have to change their existing advertising style/format or sacrifice advertising space for insertion of a meaningless QR code");
associating the recognized real-world [[ stadium seat ]] object with an interactive media record in response to the recognizing (Wan, Paragraph [0100], "The database query matches 185 a unique result in the database 35, 51, and the associated AR content 40 is retrieved 186");
presenting an interactive media item for visual presentation, including superimposing the interactive media item onto a real time display of an image captured by the portable camera to appear as if the superimposed interactive media item were part of a real world scene captured by the portable camera even though the superimposed interactive media item is virtual (Wan, Paragraph [0084], "the AR content 40 is a menu of buttons 40A, 40B, 40C <read on interactive media item> as depicted in FIG. 4 displayed within a border 40 positioned proximal to the detected text 41 in the live video feed 49"; [0014] "wherein the retrieved associated content <read on interactive media item> is superimposed in the form of Augmented Reality (AR) content on the live video feed using the display module");
PNG
media_image1.png
528
642
media_image1.png
Greyscale
enabling user interaction with the superimposed interactive media item (Wan, Paragraph [0084], "When a button 40A, 40B, 40C is pressed by the user, at least one web page is opened automatically");
[[ triggering purchasing and/or shopping from the stadium seat object in response to received user interaction with the superimposed interactive media item ]].
However, Wan does not explicitly disclose the object as a real-world stadium seat object, recognition without decoding any optical code or optical marker, or triggering purchasing and/or shopping from the stadium seat object in response to received user interaction with the superimposed interactive media item.
Nurmi teaches enabling capture by a portable camera of an image of a real-world stadium seat object having optically-readable indicia disposed thereon (Nurmi, Paragraph [0038], "The UE 101 may also execute an application 109 (e.g., a camera application or other imaging application) that can capture images"; [0067], "depicts a previously captured image 561 of stadium seating <read on image of a real-world stadium seat object>");
using at least one processor, recognizing the real-world stadium seat object based on the optically-readable indicia from the captured image (Nurmi, Paragraph [0056], "a process for recognizing objects in media content"; [0067], "depicts a previously captured image 561 of stadium seating"; "The route rendering platform 103 can interpret the stationary status as indicating that the seat is most likely the user's seat within the stadium and marks that seat with a star symbol <read on recognizing the real-world stadium seat object>");
associating the recognized real-world stadium seat object with an interactive media record in response to the recognizing (Nurmi, Paragraph [0018], "a process for associating renderings of route information rendering to image media <read on associating the recognized real-world stadium seat object with an interactive media record>"; [0059], "the user can be presented with a rendering of any routes (e.g., the user's routes) that are associated with the location in the camera's field of view").
Nurmi and Wan are analogous since both are directed to mobile device applications that capture real-world scenes using a camera and superimpose AR content onto the live image for user consumption. Wan provides a platform for recognizing text from physical objects in a live video feed and dynamically superimposing interactive AR button menus onto the display in direct response to that recognition. Nurmi provides a system for recognizing objects in image media, specifically including stadium seating, and associating those recognized objects with rendered AR content for display on a mobile device. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the stadium seat object recognition of Nurmi into the modified platform of Wan, such that the AR system is capable of recognizing physical venue objects including stadium seats and superimposing interactive AR content in response thereto, thereby extending the utility and applicability of Wan's AR platform to venue-based environments.
However, the combination of Wan, Nurmi does not explicitly disclose without decoding any optical code or optical marker or triggering purchasing and/or shopping from the stadium seat object in response to received user interaction with the superimposed interactive media item.
Mayerle teaches using at least one processor, recognizing the real-world stadium seat object based on the optically-readable indicia from the captured image without decoding any optical code or optical marker (Mayerle, Paragraph [0091], "image recognition such as for example random sample consensus or RANSAC analysis can be combined with geo-location data or other data indicating a position of a mobile device to estimate a position and pose of a camera device associated with the mobile device <read on recognizing the real-world stadium seat object based on the optically-readable indicia from the captured image without decoding any optical code or optical marker>"; [0095],"Computer vision algorithms, for example using six degrees of freedom, can be used for estimating the pose of the mobile device camera relative to the real world model (e.g., RANSAC)"); enabling user interaction with the superimposed interactive media item (Mayerle, Paragraph [0006], "a selection of the augmented reality marker is received via a user input on the screen <read on enabling user interaction with the superimposed interactive media item>, and at 1210 an item page corresponding to the physical item is presented on the screen of the mobile device in response to the received selection"; “The augmented reality view includes the two dimensional image and the two dimensional overlay presented together such that the augmented reality marker is associated with the physical item in the augmented reality view.”); triggering purchasing and/or shopping from the [[ stadium seat ]] object in response to received user interaction with the superimposed interactive media item (Mayerle, Paragraph [0007], "user interface functionality for adding the physical item or the good or service represented by the physical item to a shopping cart or watch list, user interface functionality for immediately purchasing the physical item or the good or service represented by the physical item <read on triggering purchasing and/or shopping from the stadium seat object in response to received user interaction with the superimposed interactive media item>"; [0031], " a consumer user can use the mobile device 102 to identify and select one or more items or items representative of goods or services from a shopping display 104 and thereby, among other possible functions, assign these selections to a watch list or shopping cart, purchase one or more of the selections, or the like").
Mayerle and Wan are analogous since both are dealing with references are directed to mobile device applications that leverage camera-based image capture to recognize physical real-world objects and superimpose interactive AR content onto a display for a user. Wan provide the foundation of camera-based text/object recognition and AR overlay in a venue context. Mayerle further provides a system in which the AR recognition of a physical object using computer vision techniques without relying on decoding a QR code or optical marker which directly triggers an e-commerce purchasing transaction through user interaction with the superimposed AR marker. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the computer-vision-based recognition and AR-triggered purchasing functionality taught by Mayerle into the modified invention of Wan such that when a physical venue object such as a stadium seat is recognized from a live camera image, the resulting superimposed interactive AR media item enables a user to trigger a purchasing or shopping transaction directly from that recognized seat object, thereby enhancing the commercial utility of the AR system and satisfying the consumer demand for venue-based mobile purchasing identified in the art. The motivation is to provide users with the ability to make immediate purchase decisions at the point of visual recognition of a physical item.
Regarding Claim 4, the combination of Wan, Nurmi, and Mayerle teaches the invention in Claim 1.
The combination further teaches the [[ stadium seat ]] object has [[ optical ]] indicia printed thereon (Wan,Paragraph[0077], "The text markers 80 in the live video feed 49 for detection by the OCR engine 32 may be found on printed or displayed matter 70 <read on indicia printed thereon>, for example, outdoor advertising, shop signs, advertising in printed media"), and the recognizing comprises recognizing at least some of the printed [[ optical ]] indicia (Wan, Paragraph [0076], "The OCR engine 32 detects the presence of text in a live video feed 49 captured by the built-in device video camera 21 in real-time, and converts the detected text 41 into machine-encoded text in real-time <read on recognizing comprises recognizing at least some of the printed indicia>"; [0035], "The text markers may include any one from the group consisting of spaces, edges, colour, and contrast <read on indicia printed thereon>").
But Wan does not explicitly disclose that the printed object is a stadium seat object, nor does it explicitly label the indicia as optical indicia.
However, Nurmi teaches the stadium seat object has [[optical]] indicia printed thereon (Nurmi,Paragraph[0067], "FIG. 5D also depicts a previously captured image 561 of stadium seating <read on stadium seat object>. In this example, the stadium is equipped with an indoor positioning system with accuracy down to the seat level <read on stadium seat object>"; Nurmi,Paragraph[0068], "a user can use an external camera view or video stream e.g., provided by the audio-visual facilities of the stadium with the users mobile device and see on that image the users route towards his or her seat <read on stadium seat object>").
Nurmi and Wan are analogous since both are directed to systems that process images of real-world physical environments using a mobile device camera and present associated information overlaid on a display for the benefit of a mobile device user. Wan provided a way of recognizing printed text markers from physical real-world objects using an OCR engine operating on a live video camera feed, and superimposing AR content on the display in response to that recognition. Nurmi provided a way of identifying and processing stadium seat objects from captured image media, with positional accuracy down to the individual seat level within a stadium environment. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the stadium seat object context taught by Nurmi into the modified invention of Wan such that the object being recognized from the live video feed is a stadium seat object, thereby satisfying the limitation that the stadium seat object has indicia printed thereon and the recognizing comprises recognizing at least some of the printed indicia. The motivation is to enable a mobile device user to retrieve seat-specific augmented reality information by simply pointing a mobile device camera at a stadium seat.
But Wan does not explicitly disclose that the indicia are optical indicia.
However, Mayerle teaches the stadium seat object has optical indicia printed thereon, and the recognizing comprises recognizing at least some of the printed optical indicia (Mayerle,Paragraph[0034], "identification data e.g. barcodes, QR codes, or the like <read on optical indicia printed thereon> relating to products e.g. goods, services, etc."; [0040], "The location code can optionally include an image of a quick response QR code <read on optical indicia printed thereon> or some other scannable pattern or design captured by the camera of the mobile device 102 <read on recognizing comprises recognizing at least some of the printed optical indicia>"; [0109], "Other possible input devices include, but are not limited to, touch screens or other touch-sensitive devices such as single or multi-point resistive or capacitive trackpads, voice recognition hardware and software, optical scanners <read on optical indicia>, optical pointers, digital image capture devices and associated interpretation software <read on recognizing comprises recognizing at least some of the printed optical indicia>, and the like").
Mayerle and Wan are analogous since both are directed to systems that use a mobile device camera to capture images of physical objects bearing printed identifying marks, and retrieve associated augmented reality content to be displayed to the user on a mobile device screen. Wan provided a way of recognizing printed text markers from physical real-world objects using an OCR engine operating on a live video camera feed, and superimposing AR content on the display in response to that recognition. Mayerle provided a way of using optically scannable indicia specifically barcodes and QR codes printed on physical objects as identifiers to be captured by a mobile device camera and used to retrieve associated augmented reality content. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the optically scannable printed indicia taught by Mayerle into the modified invention of Wan such that optically scannable indicia such as barcodes, QR codes, seat numbers, row letters, and section identifiers printed on a stadium seat object would be recognized by the modified system from the captured live video feed, thereby satisfying the limitation that the stadium seat object has optical indicia printed thereon and the recognizing comprises recognizing at least some of the printed optical indicia. The motivation is to provide a reliable and machine-readable optical identification mechanism on the stadium seat object that enables fast and accurate recognition by the mobile device camera, as Mayerle expressly discloses that printed optical indicia such as barcodes and QR codes serve as the identification mechanism for linking a physical object to its associated augmented reality content.
Regarding Claim 5, the combination of Wan, Nurmi and Mayerle teaches the invention in Claim 4.
The combination further teaches wherein the recognizing includes recognizing , using at least one processor in a network, characters printed on the [[ stadium seat ]] object (Wan, Paragraph [0010], “there is provided a platform for recognising text using mobile devices with a built-in device video camera and automatically retrieving associated content based on the recognized text”; [0044], “a processor executing computer-readable instructions on a mobile device that, when executed by the processor, cause the processor to perform”)
[[ permitting a remote wearable and/or holdable device bearing the portable camera ]]
to access, over an electronic wireless network, a database entry or [[ record stored in the cloud ]] and matched to the recognized characters printed on the [[ stadium seat ]] object (Wan, Paragraph [0016], "The database may be stored on the mobile device, or remotely stored and accessed via the Internet"; [0079], “The mobile device 20 includes … Basic hardware requirements ofthe mobile device 20 include a: video camera 21, WiFi and/or 3G data connectivity 22”; [0100], “feed 49, it converts (183) it into machine-encoded text and a query (184) on the database 35, 51 is performed. The database query matches (185) a unique result in the database 35, 51, and the associatedAR content 40 is retrieved”).
But Wan does not explicitly disclose [[ recognizing, using at least one processor in a network, characters printed on the ]] stadium seat [[ object ]]. permitting a remote wearable and/or holdable device bearing the portable camera.
However, Nurmi teaches recognizing, using at least one processor in a network, characters printed on the stadium seat object (Nurmi, Paragraph [0067], “depicts a previously captured image 561 of stadium seating” [0025], “the images can be presented in two-dimensions or three-dimensions” [0061], “rendering platform 103 performs the process 400 and is implemented in, for instance, a chip set including a processor” [0067], “The route rendering platform 103 can interpret the stationary status as indicating that the seat is most likely the user's seat within the stadium and marks that seat with a star symbol” [0070], “digital data that is used to represent a number or code for a character”) permitting a remote wearable and/or holdable device bearing the portable camera (Nurm, Paragraph [0045], "The UE 101 is any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, station, unit, device, multimedia computer, multimedia tablet, Internet node, communicator, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, wearable device, personal communication system (PCS) device, personal navigation device, personal digital assistants (PDAs), audio/video player, digital camera/camcorder...positioning device, wearable device, head-mounted device...It is also contemplated that the UE 101 can support any type of interface to the user (such as 'wearable' circuitry, etc.)"
As explained in rejection of claim 1, the obviousness for combining of stadium seat object of Nurmi into Wan is provided above.
However, the combination of Wan, Nurmi does not explicitly access data over record stored in the cloud.
However, Mayerle teaches mitting a remote wearable and/or holdable device bearing the portable camera to access data over record stored in the cloud (Mayerle, Paragraph [0023], "A local client shopping application...can execute on the mobile device to provide a consumer user of the mobile device with real time information regarding goods or services...through leveraging an enterprise resource planning (ERP) or other business software solution backend, which can be accessed over a network (e.g. via cloud computing or the like).
Wan, Nurmi, and Mayerle are analogous prior art, as all three references are directed to mobile-device-based systems that use a portable camera and wireless network connectivity to capture images, perform recognition, and retrieve associated content from a remote backend database for display on the mobile device. Wan provides a foundational OCR-based recognition platform using a server processor over a network to recognize printed characters from a live video feed and retrieve matched AR content from a remotely accessible database. Nurmi provides the stadium venue context, including recognition of stadium seat objects and the express teaching that the user device bearing the camera may be a wearable or head-mounted device. Mayerle provides the explicit teaching of cloud-based backend database storage accessible over a wireless network, including matched retrieval of database records corresponding to recognized objects. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the cloud-based backend database architecture of Mayerle into the combined invention of Wan, Nurmi system, because cloud-based storage was a well-known and widely adopted implementation for remotely accessible databases at the time of the invention, and doing so would improve the scalability and accessibility of the matched content database of Wan's recognition platform without any unpredictable results. The motivation to combine is further supported by Mayerle's explicit disclosure that such cloud-accessible backends can be used in conjunction with mobile devices executing augmented reality applications over wireless networks.
Regarding Claim 6, The combination of Wan, Nurmi, and Mayerle teaches the invention in Claim 5 as set forth above.
The combination further teaches wherein the optically-readable indicia on the stadium seat object comprises a patch of printed material attached to the stadium seat object.
Wan does not explicitly disclose that the printed material is a patch attached to a stadium seat object.
However, Nurmi teaches the [[ optically-readable indicia on the ]] stadium seat object comprises a patch of printed material attached to the stadium seat object (Nurmi, Paragraph [0029], "the system 100 can attach the route information or rendering the image media directly as metadata <read on attached to>; the route information or rendering may be maintained as a separate file, and then the file can be associated e.g., via a timestamp, index, etc. with the corresponding image media <read on patch of printed material attached to the stadium seat object>"; Nurmi, ¶0067, "FIG. 5D also depicts a previously captured image 561 of stadium seating <read on stadium seat object>").
But the combination of Wan, Nurmi, and Mayerle does not explicitly disclose that the indicia constitute a physically separate patch of printed material that is attached to the stadium seat object.
However, Mayerle teaches the optically-readable indicia on the [[ stadium seat ]] object comprises a patch of printed material attached to the stadium seat object (Mayerle, Paragraph [0088], "the consumer user can scan a QR-code sticker <read on patch of printed material> in or on the shopping display 104 <read on attached to the stadium seat object> for example on a window in front of the shopping display 104 to identify the shopping display 104 <read on printed material attached to an object>"; [0034], "identification data e.g. barcodes, QR codes, or the like <read on optically-readable indicia> relating to products e.g. goods, services, etc.").
Mayerle and Wan are analogous since both are directed to systems that use a mobile device camera to capture and recognize printed indicia physically attached to or displayed on a real-world object, in order to retrieve and superimpose associated augmented reality content on a display. Wan provided a way of recognizing printed text markers found on physical objects via a mobile device camera and OCR engine, and superimposing AR content on the live video feed in response). Mayerle provided a way of using a scannable sticker patch, specifically a QR code sticker attached to a physical object as the printed optical indicia identifier that links the physical object to its associated augmented reality content on a mobile. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the attached printed patch indicia taught by Mayerle into the modified invention of Wan, such that the optically-readable indicia on the stadium seat object comprises a patch of printed material such as a QR code sticker or label physically attached to the stadium seat object and recognized by the mobile device camera, thereby satisfying the limitation that the optically-readable indicia on the stadium seat object comprises a patch of printed material attached to the stadium seat object. The motivation is to provide a practical and standardized means of affixing machine-readable optical indicia to a stadium seat object without requiring permanent printing on the seat itself, as Mayerle expressly teaches that attaching a scannable QR-code sticker to a physical object is a convenient and reliable mechanism for linking that object to its augmented reality content).
Regarding Claim 7, the combination of Wan, Nurmi and Mayerle teaches the invention in Claim 6.
The combination further teaches the interactive media item comprises a digital overlay (Wan, Paragraph [0084],"the AR content 40 is a menu of buttons 40A, 40B, 40C as depicted in FIG. 4 displayed within a border 40 positioned proximal to the detected text 41 in the live video feed 49"; [0014], “wherein the retrieved associated content is superimposed in the form ofAugmented Reality (AR) content on the live video feed using the display module”) that leads to specific marketing action relating to the recognized stadium seat object" (Wan, Paragraph [0084], “When a button 40A, 40B, 40C is pressed by the user, at least one web page is opened automatically” “if the “Reviews” button 40A is pressed, the web page that is automatically opened” “which is web page containing user reviews of the restaurant on the Open Rice web site”; [0136], AR content 40 for Discount will link to AR link Groupon, Credit Card Discounts; [0065], "the platform extends the advertising reach of businesses without requiring them to modify their existing advertising style, and increases their brand awareness to their target market by linking the physical world to their own generated digital content"); providing specific marketing information; a video, tutorial, or displayable marketing content relating to the recognized stadium seat object" (Wan, Paragraph [0084], "the web page or digital content from the URL can be displayed in-line as AR content 40...For example, a video from YouTube can be streamed or a PDF file can be downloaded and displayed by the display module 31 and are superimposed on the live video feed 49, or an audio stream is played to the user while the live video feed 49 is active. Both the video and audio stream may be review or commentary about the restaurant"; [0087], "videos hosted on YouTube with content related to the machine-encoded text, review sites related to the machine-encoded text, Facebook updates containing the machine-encoded text, Twitter posts containing the machine-encoded text, discount coupon sites containing the machine-encoded text"; [0091], "businesses...upload an image to the server 50 of their latest advertisement, shop sign or logo and associate it with machine-encoded text and a URL...gives them greater control to operate their own marketing, if they change the URL to a web page for their current advertising campaign")
Mayerle further teaches the interactive media item comprises a digital overlay (Mayerle, Paragraph [0006], "determining, from an augmented reality three dimensional model of a shopping display, a two dimensional overlay that includes an augmented reality marker corresponding to a physical item visible in the two dimensional image"; “displaying, via a screen on a mobile device, an augmented reality view of the shopping display"; "augmented reality view includes the two dimensional image and the two dimensional overlay presented together such that the augmented reality marker is associated with the physical item in the augmented reality view") that leads to specific marketing action relating to the recognized [[ stadium seat ]] object (Mayerle, Paragraph [0031], "a consumer user can use the mobile device 102 to identify and select one or more items or items representative of goods or services from a shopping display 104 and thereby, among other possible functions, assign these selections to a watch list or shopping cart, purchase one or more of the selections, or the like"; [0006], "the item page includes additional information about at least one of the physical item and a good or service represented by the physical item"; [0007], “user interface functionality for immediately purchasing <read on marketing action> the physical item or the good or service represented by the physical item, user interface functionality that supports sending information about the physical item or the good or service represented by the physical item to another person”) selected from the group consisting of providing specific marketing information; a video, tutorial, or displayable marketing content relating to the recognized [[ stadium seat ]] object (Mayerle, Paragraph [0024], "augmented reality is a live, direct or indirect, view of a physical shopping window whose products are augmented by computer-generated virtual objects like markers, additional product information or dotted lines around real world product"; [0043], “The information screen, which can also be referred to as an item page, can optionally include one or more of identification information relating to the selected item 302; sizing information…user interface functionality that supports viewing of related items or suggested items”).
Mayerle and Wan are analogous since both are directed to systems that use a mobile device camera to recognize physical objects and superimpose interactive digital content on a display in response to that recognition. Wan provided a way of superimposing a menu of interactive AR buttons on a live video feed in response to recognizing a physical object, where each button triggers a specific action. Mayerle provided a way of presenting augmented reality digital overlays specifically designed to lead the user to marketing actions such as viewing product information, adding items to a shopping cart, purchasing immediately, and sharing marketing content all triggered in response to recognizing a physical object from a mobile device camera. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the marketing-specific digital overlay actions taught by Mayerle into the modified invention of Wan, such that the interactive media item superimposed in response to recognizing the stadium seat object is a digital overlay that leads to a specific marketing action such as providing marketing information, a video, tutorial, or displayable marketing content relating to the recognized stadium seat object. The motivation is to commercialize the augmented reality recognition system by enabling seat-specific marketing actions directly from the recognized stadium seat object, as Mayerle expressly teaches that linking physical object recognition to marketing overlays drives sales and improves the consumer shopping experience
Regarding Claim 8, the combination of Wan, Nurmi, and Mayerle teaches the invention in Claim 1.
The combination further teaches the superimposing is performed on a [[ display provided by smart glasses ]] (Wan, Paragraph [0013], "a mobile application executed by the mobile device, the mobile application including a display module for displaying the live video feed on a screen of the mobile device <read on display>"; [0014], "wherein the retrieved associated content is superimposed in the form of Augmented Reality AR content on the live video feed using the display module <read on superimposing is performed on a display>"; [0079), "The mobile device 20 includes a smartphone such as an Apple iPhone, or a tablet computer such as an Apple iPad <read on display device>").
But Wan does not explicitly disclose that the display is a display provided by smart glasses.
However, Nurmi teaches the superimposing is performed on a display provided by smart glasses (Nurmi, Paragraph [0038], "The UE 101 is any type of mobile terminal, fixed terminal, or portable terminal <read on display provided by smart glasses>"; [0025], "the images can be presented in two-dimensions or three-dimensions <read on display>”; It is noted that Nurmi's disclosure of "any type of mobile terminal, fixed terminal, or portable terminal which inherently encompasses smart glasses, as smart glasses are a well-recognized species of portable terminal in the augmented reality field. Smart glasses are a wearable portable device bearing a display and a camera — both of which are expressly required by Nurmi's system — and therefore fall squarely within the scope of Nurmi's "any type of portable terminal" disclosure. One of ordinary skill in the art at the time of the invention would have recognized smart glasses as a portable terminal capable of executing the AR superimposition described by Nurmi).
Nurmi and Wan are analogous since both are directed to systems that superimpose augmented reality content onto a display of a portable device in response to recognizing a real-world object from a camera feed. Wan provided a way of superimposing AR content on a mobile device display in real-time in response to object recognition, using a display module of a portable device. Nurmi provided a way of superimposing route and location overlay information onto a display of any type of portable terminal operating in a stadium environment, including wearable portable devices. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to implement the superimposing of the interactive media item onto a display provided by smart glasses i.e. a portable terminal bearing a display and camera well known in the AR field at the time of the invention within the modified system of Wan, thereby satisfying the limitation that the superimposing is performed on a display provided by smart glasses. The motivation is to provide a hands-free, wearable display option for stadium users viewing AR content triggered by a recognized stadium seat object, as Nurmi expressly contemplates deployment of the AR system on any type of portable terminal.
Regarding Claim 10, The combination of Wan, Nurmi, and Mayerle teaches the invention in Claim 1.
The combination further teaches displaying any or all of the following action buttons in any combination or subcombination: Price Tag, Photo Gallery of purchasable products or services, Videos of purchasable products or services, Description of purchasable products or services, Call, Shop link to buy merchandise or tickets, Map of a stadium, [[ Order food ]], and/or Booking opportunities (Wan, Paragraph [0084], "the AR content 40 is a menu of buttons 40A, 40B, 40C <read on action buttons> as depicted in FIG. 4 displayed within a border 40 positioned proximal to the detected text 41 in the live video feed 49"; [0136], "AR content 40 AR Link Name of the food Reviews Openrice <read on action buttons relating to food> Share Facebook, Twitter Discounts Groupon, Credit Card Discounts <read on Discount Codes> Star rating Zagat <read on Description> Name of shop Reviews Fodors, TripAdvisor Share Facebook, Twitter Discounts Groupon, Credit Card Discounts Shops Advertising Shops URL, YouTube campaign <read on Videos, Shop link> Research Wikipedia <read on Description>"; Fig. 4, Element 40A, Paragraph [0084], " the AR content 40 is a menu of buttons 40A, 40B, 40C as depicted in FIG. 4… if the “Reviews” button 40A is pressed, the web page that is automatically opened <read on Description of purchasable products or services>"; Fig. 4, Element 40B, Discount button <read on Discount Codes>"; [0087], "Other URLs linked to AR content 40 include videos hosted on YouTube with content related to the machine-encoded text <read on Videos>, review sites related to the machine-encoded text <read on Description>, Facebook updates <read on Photo Gallery>, discount coupon sites <read on Discount Codes>").
But Wan does not explicitly disclose action buttons specifically for Order food or a Map of a stadium in the context of a stadium seat object.
However, Nurmi teaches the Map of a stadium action button (Nurmi, Paragraph [0059], " the user can be presented with a rendering of any routes (e.g., the user’s routes) that are associated with the location in the camera’s field of view <read on Map of a stadium>"; [0067], "FIG. 5D also depicts a previously captured image 561 of stadium seating <read on stadium context>. In this example, the stadium is equipped with an indoor positioning system with accuracy down to the seat level <read on Map of a stadium>").
Nurmi and Wan are analogous since both are directed to systems that process images of real-world environments and present associated information overlaid on a display for the benefit of a mobile device user. Wan provided a way of superimposing a menu of interactive AR action buttons on a live video feed in response to recognizing a physical object. Nurmi provided a way of presenting stadium-specific route and mapping information overlaid on a captured image of a stadium environment, including navigation to a specific seat. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the stadium mapping and routing information taught by Nurmi into modified invention of Wan such that a Map of a stadium action button is included among the displayed action buttons, thereby satisfying that limitation. The motivation is to provide stadium users with a convenient and directly accessible navigation tool from the AR interface triggered by the recognized stadium seat object, enabling users to find their seat or navigate within the stadium.
Mayerle teaches the Order food action button (Mayerle, Paragraph [0026], "a local client shopping application 110 that can support functionality such as a shopping basket or cart <read on Shop link to buy merchandise>, a user purchasing profile e.g. a shopper identity, payment information, delivery address <read on Booking opportunities>, item selection of goods or services represented in a shopping display 104 <read on Shop link>"; [0043], "user interface functionality for adding the item optionally with additional sizing, quantity, color, shape, etc. information to a shopping cart, watch list, or the like or, alternatively for buying the item immediately without use of a shopping cart <read on Shop link to buy merchandise>"; [0032], "it can be advantageous for a local client on a mobile device 102 to be able to accurately inform a consumer user of the current, actual pricing associated with selection made by the consumer user via the user interface of the mobile device <read on Price Tag>. Additionally, to enable completion of a sale at the time that a consumer user making a buying decision, an availability status of a specific good or service e.g. a specific size of shoe, an open time slot for a haircut or massage <read on Booking opportunities> can be advantageously available in real time"; [0048], "selection of a buy button or similar functionality 702 by the consumer user can cause the user interface to display a cart addition screen 704, in which a consumer user can choose to add or not add one or more items to a virtual shopping cart for current or later purchase <read on Order food, Booking opportunities>").
Mayerle and Wan are analogous since both are directed to systems that superimpose interactive AR action buttons onto a real-world camera view in response to recognizing a physical object, enabling a user to complete purchasing and ordering transactions directly from the AR interface on a mobile device. Wan provided a way of displaying a menu of interactive AR action buttons on a live video feed triggered by object recognition, where each button links to a specific action or URL. Mayerle provided a way of presenting a comprehensive suite of commerce-oriented interactive action buttons — including ordering, purchasing, price checking, and booking — as part of an augmented reality shopping interface triggered by recognition of a physical object. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the commerce-oriented action buttons taught by Mayerle into modified invention of Wansuch that the displayed action buttons include any or all of: Price Tag, Photo Gallery, Videos, Description, Call, Shop link to buy merchandise or tickets, Map of a stadium, Discount Codes, Order food, and/or Booking opportunities e.t.c. The motivation is to expand the utility of the AR interface triggered by the recognized stadium seat object to encompass the full range of commerce and service actions available to a stadium user including ordering food and booking additional services directly from the AR overlay
Regarding Claim 11, it recites limitations similar in scope to the limitations of claim 1, but in a system. As shown in the rejection, the combination of Wan, Nurmi, and Mayerle disclose the limitations of claims 1. Additionally, Wan discloses an system that maps to Fig. 1 and Paragraph [0008], [0054], [0080], (Wan, Paragraph [0008], [0080], “The mobile application 30 is run on a mobile operating system” “a processor to execute computer-readable instructions to perform” “A mobile application called Google™ Goggles analyses a still image captured by a camera phone”). Thus, Claim 11 is met by Wan according to the mapping presented in the rejection of claims 1, given the method corresponds to the system.
Regarding Claim 14, it recites limitations similar in scope to the limitations of Claim 4 with additional limitation without decoding any optical code or optical marker which taught by Wan without decoding any optical code or optical marker (Wan, Paragraph [0007], "However, this process is cumbersome as it requires businesses to work closely with the advertising agency in order to place the QR code at a specific position of the outdoor advertisement. This wastes valuable advertising space, and the QR code only serves a single purpose to small percentage of passer bys and therefore has no significance to the majority of passer bys"; [0086], "they do not have to change their existing advertising style/format or sacrifice advertising space for insertion of a meaningless QR code") and therefore is rejected under the same rationale.
Regarding Claim 15, the combination of Wan, Nurmi and Mayerle teaches the invention in Claim 14.
The combination further teaches wherein the recognizing includes recognizing characters printed on the [[ stadium seat ]] object (Wan, Paragraph [001 0], "there is
provided a platform for recognising text using mobile devices with a built-in device video camera and automatically retrieving associated content based on the recognized text") without decoding any optical code or optical marker (Wan, Paragraph [0007], "However, this process is cumbersome as it requires businesses to work closely with the advertising agency in order to place the QR code at a specific position of the outdoor advertisement. This wastes valuable advertising space, and the QR code only serves a single purpose to small percentage of passer bys and therefore has no significance to the majority of passer bys"; [0086], "they do not have to change their existing advertising style/format or sacrifice advertising space for insertion of a meaningless QR code").
However, Wan does not explicitly disclose printed on the stadium seat object.
But Nurmi teaches recognizing characters printed on the stadium seat object (Nurmi, Paragraph [0067], "depicts a previously captured image 561 of stadium
seating" [0025], "the images can be presented in two-dimensions or threedimensions" [0067], "The route rendering platform 103 can interpret the
stationary status as indicating that the seat is most likely the user's seat within
the stadium and marks that seat with a star symbol" [0070], "digital data that is
used to represent a number or code for a character")
As explained in rejection of claim 1, the obviousness for combining of stadium
seat object of Nurmi into Wan is provided above.
Regarding Claim 16, it recites limitations similar in scope to the limitations of Claim 6 and therefore is rejected under the same rationale.
Regarding Claim 17, it recites limitations similar in scope to the limitations of Claim 7 and therefore is rejected under the same rationale.
Regarding Claim 20, it recites limitations similar in scope to the limitations of Claim 10 and therefore is rejected under the same rationale.
Claim(s) 2, 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wan (US 20140111542 A1), in view of Nurmi (US 20120221241 A1), further in view of Mayerle et al (US 20140100997 A1, hereinafter " Mayerle") as applied to Claim 1 above and future in view of Bennett et al. (US 20200273254 A1, hereinafter Bennett).
Regarding Claim 2, the combination of Wan, Nurmi, and Mayerle teaches the invention in Claim 1.
The combination further teaches wherein the superimposing comprises using at least one of augmented reality, mixed reality and virtual reality (Wan, Paragraph [0014], “wherein the retrieved associated content is superimposed in the form of Augmented Reality (AR) content on the live video feed using the display module”; it is noted ).
However, The combination does not explicitly disclose [[ superimposing comprises using at least one of augmented reality, ]] mixed reality and virtual reality.
But Bennett teaches the superimposing comprises using at least one of augmented reality, mixed reality and virtual reality (Bennett, Paragraph [0089], “an operator 680 is wearing mixed-reality (MR) device 601. Mixed-reality device 601 is an example of hologram device 501 that is a wearable, head-mounted display mixed-reality device. Via MR device 601, in the example illustrated in FIG. 6, the operator can see step card 671, picture 672, 3D hologram 673, and tether 674, all superimposed on a real-world environment” [0002], “mixed reality takes place not only in the physical world or the virtual world, but includes a mix of elements from reality and virtual reality, encompassing both augmented reality and augmented virtuality via immersive technology”).
Bennett and Wan are analogous since both of them are dealing with processing data in augmented reality environment. Wan provided a way of recognized object from image and superimposing action button on the image when dealing with data in the augmented reality environment. Bennett provided a way of recognized object from image and superimposing action button on the image when dealing with data in the not only mixed reality environment but also augmented reality and virtual environment. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate multiple environments taught by Bennett into modified invention off Wan such that when dealing with data in the three dimension world, system will be able to running not only just augmented reality environment but also support multiple environments like mixed rality, virtual reality as well which enhance the capability of the system and provide more user friendly experience.
Regarding Claim 12, it recites limitations similar in scope to the limitations of Claim 2 and therefore is rejected under the same rationale.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wan (US 20140111542 A1), in view of Nurmi (US 20120221241 A1), further in view of Mayerle et al (US 20140100997 A1, hereinafter " Mayerle") as applied to Claim 1, 11 above respectively and further in view of Neumann et al. (US 20190220665 A1, hereinafter Neumann) and Bjontegard (US 20150262208 A1).
Regarding Claim 3, the combination of Wan, Nurmi and Mayerle teaches the invention in Claim 1.
The combination further teaches wherein recognizing comprises recognizing a three-dimensional real-world [[stadium seat]] object with a seat number printed on it (Wan, Paragraph [0077], "The text markers 80 in the live video feed 49 for detection by the OCR engine 32 may be found on printed or displayed matter 70, for example, outdoor advertising, shop signs, advertising in printed media, or television or dynamic advertising light boxes").
However, Wan does not explicitly disclose a [[stadium seat]] object with a seat number printed on it.
But Nurmi teaches recognizing a [[three-dimensional]] stadium seat object with [[a seat number]] printed on it (Nurmi, Paragraph [0066], "depicts a previously captured image 561 of stadium seating"; [0025], "the images can be presented in two-dimensions or three-dimensions"; [0067], "The route rendering platform 103 can interpret the stationary status as indicating that the seat is most likely the user's seat within the stadium and marks that seat with a star symbol" <read on object with print>).
As explained in the rejection of Claim 1, the obviousness for combining the stadium seat object of Nurmi into Wan is provided above.
However, the combination does not explicitly disclose the three-dimensional real-world stadium seat object with a seat number printed on it.
But Neumann teaches the three-dimensional real-world stadium seat object with a seat number printed on it (Neumann, Paragraph [0007], "user ticketing information, to guide the user in navigating to another part of the venue, e.g. the user's seat"; [0005], "text from an object captured in the digital image is recognized, e.g., using text recognition and object recognition. The text, in this example is indicative of a location, e.g., a sign indicating a corresponding section in a stadium, which is used to directly determine a location with respect to a digital map"; [0018], "receives a digital image, a digital ticket, and 2D and 3D maps"; [0025], "text associated with the identified objects is identified using optical character recognition (OCR) techniques. In this example, the text may identify a current location directly, e.g., indicate a section of the physical venue, seat location, and so forth"; [0038], "The AR digital content 126, for instance, may describe a location of a seat, directions to the seat, a relation of that seat to other seats, directions to desired services available at the physical environment 106"; Paragraph , "data describing where such access is permitted (e.g., suite number, seat number, section number, level, parking spot, field access)"). includes determining location of the three-dimensional real-world stadium seat object within a stadium, concert hall, theater or other venue in response to the recognizing (Neumann, Paragraph [0005], "text from an object captured in the digital image is recognized, e.g., using text recognition and object recognition. The text, in this example is indicative of a location, e.g., a sign indicating a corresponding section in a stadium, which is used to directly determine a location with respect to a digital map"; [0025], "the text may identify a current location directly, e.g., indicate a section of the physical venue, seat location, and so forth"; [0050] , "A location of the object 302 in relation to a digital map of a physical environment is then determined by the location determination system 120"; [0055] , "the location determination system 120 may leverage text that directly indicates a location (e.g., a section, seat number)").
Neumann and Wan are analogous since both are directed to processing and presenting interactive data in an augmented reality environment applied to physical venue spaces. Wan provides a way to recognize text markers on physical objects from a live camera feed and superimpose interactive AR content in response. Neumann provides a way to recognize text or indicia from an object captured in a digital image in a stadium or venue context and use that recognized text to directly determine the physical location of that seat within the venue, and further leverages that determined location to provide contextual venue services near the seat. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the seat-location determination taught by Neumann into the modified combination of Wan, such that when the system recognizes printed text on a three-dimensional stadium seat object, it uses that recognized text to directly determine the physical location of that seat within the venue, which enhances the capability of the AR system by providing the user with precise seat location awareness as an immediate and predictable result of the recognition step already performed by Wan.
However, the combination of Wan, Nurmi, and Neumann does not explicitly disclose but Bjontegard teaches initiating delivery of an ordered or purchased item to the determined seat location (Bjontegard, Paragraph [0097], "As the fan leaves the stadium, the fan's connected device will cross the established geo-fence and a thank you for coming message can be displayed. This can include an interactive button with an offer and a call to action such as 'come back Thursday night--buy now and get 2 tickets for the price of one'" <read on initiating an order/purchase action>; [0198], "This complete solution thereby enables the stadium owner and/or team to communicate with their fans from the moment they purchase their tickets, as they are coming to the stadium" <read on connecting a purchase action to a stadium seat context>).
Bjontegard and Wan are analogous since both are directed to delivering interactive actionable purchase triggers to a user in the context of a stadium or live event environment. Wan provides a way to superimpose interactive action buttons — including purchase-enabling buttons — onto a live camera feed in response to recognizing physical object indicia. Bjontegard provides a way to deliver interactive call-to-action buttons tied to the stadium seat context that initiate ordering and purchase transactions connected to a specific fan's seat location within the stadium. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the delivery-to-seat purchase initiation taught by Bjontegard into the modified combination of Wan, Nurmi, and Neumann, such that upon the triggered purchasing and/or shopping step the system initiates delivery of the ordered or purchased item to the determined physical seat location, which closes the loop of the purchase transaction by routing delivery to the seat location already determined from the recognition step and provides a more complete and user-friendly stadium seat experience.
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wan (US 20140111542 A1), in view of Nurmi (US 20120221241 A1), further in view of Mayerle et al (US 20140100997 A1, hereinafter " Mayerle") as applied to Claim 1, 11 above respectively and further in view of Neumann et al. (US 20190220665 A1, hereinafter Neumann).
Regarding Claim 13, the combination of Wan, Nurmi, and Mayerle teaches the invention in Claim 11.
The combination further teaches wherein the recognition comprises recognizing a three-dimensional [[ stadium seat ]] object with a seat number printed on it (Wan, Paragraph [0077], “The text markers 80 in the live video feed 49 for detection by the OCR engine 32 may be found on printed or displayed matter 70, for example, outdoor advertising, shop signs, advertising in printed media, or television or dynamic advertising light boxes”) without decoding any optical code or optical marker (Wan, Paragraph [0007], "However, this process is cumbersome as it requires businesses to work closely with the advertising agency in order to place the QR code at a specific position of the outdoor advertisement. This wastes valuable advertising space, and the QR code only serves a single purpose to small percentage of passer bys and therefore has no significance to the majority of passer bys"; [0086], "they do not have to change their existing advertising style/format or sacrifice advertising space for insertion of a meaningless QR code");
However, Wan does not explicitly disclose stadium seat [[ object with a seat number on it ]].
But Nurmi teaches recognizing a [[ three-dimensional ]] stadium seat object with [[ a seat number ]] an object printed on it (Nurmi, Paragraph [0067], “depicts a previously captured image 561 of stadium seating” [0025], “the images can be presented in two-dimensions or three-dimensions” [0067], “The route rendering platform 103 can interpret the stationary status as indicating that the seat is most likely the user's seat within the stadium and marks that seat with a star symbol <read on object with print>”)
As explained in rejection of claim 1, the obviousness for combining of stadium seat object of Nurmi into Wan is provided above.
But the combination does not explicitly disclose the [[ three-dimensional ]] stadium seat object with [[ a seat number ]] printed on it.
However, Neumann teaches the three-dimensional stadium seat object with a seat number printed on it (Neumann, Paragraph [0007], “user ticketing information, to guide the user in navigating to another part of the venue, e.g. the user's seat” [0005], “example, text from an object captured in the digital image is recognized, e.g., using text recognition and object recognition. The text, in this example is indicative of a location, e.g., a sign indicating a corresponding section in a stadium, which is used to directly determine a location with respect to a digital map” [0018], “receives a digital image, a digital ticket, and 2D and 3D maps” [0037], “The AR digital content 126, for instance, may describe a location of a seat, directions to the seat, a relation of that seat to other seats, directions to desired services available at the physical environment 106,” [0060], “The location determination system 120 also includes access to digital images 114 captured by the digital camera 112, e.g., as part of a "live stream." Access to the digital ticket 208 is also permitted, which may include functionality usable to permit user access to the physical environment (e.g., a bar code, QR code), data describing where such access is permitted ( e.g., suite number, seat number, section number, level, parking spot, field access)”).
Neumann and Wan are analogous since both of them are dealing with processing data in augmented reality environment. Wan provided a way of recognized object from image and superimposing action button on the image when dealing with data in the augmented reality environment. Neumann provided a way of recognized object from digital medium of stadium ticket and identify the seat number from digital image. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate ticket information identification taught by Neumann into modified invention off Wan such that when dealing with data in the three dimension world, especially dealing with stadium ticket image, system will be able to identify the seat number from the ticket and easily for user to access the seat during the event in the stadium which create more user friendly access to the information in the augmented reality environment.
Claim(s) 9, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wan (US 20140111542 A1), in view of Nurmi (US 20120221241 A1), further in view of Mayerle et al (US 20140100997 A1, hereinafter " Mayerle") as applied to Claim 1 above and future in view of Bjontegard (US 20150262208 A1).
Regarding Claim 9, the combination of Wan, Nurmi, and Mayerle teaches the invention in Claim 1.
The combination further further teaches wherein the selected media item comprises a [[ call ]] button to call a service to the stadium seat object (Wan, Paragraph [0084], “the AR content 40 is a menu of buttons 40A, 40B, 40C as depicted in FIG. 4 displayed within a border 40 positioned proximal to the detected text 41 in the live video feed 49”).
But Wan does not explicitly disclose [[ wherein the selected media item comprises a ]] call [[ button ]].
However, Bjontegard teaches the selected media item comprises a call button to call a service to the stadium seat object (Bjontegard, Paragraph [0097], “As the fan leaves the stadium, the fan's connected device will cross the established geo-fence and a "thank you for coming" message can be displayed. This can include an interactive button with an offer and a call to action such as "come back Thursday night--buy now and get 2 tickets for the price of one"
[0198], “This can be an AR game such as a soccer penalty kick game, basketball free throw game, or a baseball bating versus pitcher game, basically anything that is related to the sport being played in the stadium that the ticket will provide entrance to” [0198], “This complete solution thereby enables the stadium owner and/or team to communicate with their fans from the moment they purchase their tickets, as they are coming to the stadium”).
Bjontegard and Wan are analogous since both of them are dealing with processing data in augmented reality environment. Wan provided a way of recognized object from image and superimposing action button on the image to call for ticket service when dealing with data in the augmented reality environment. Bjontegard provided a way of overlaying call button on the image while dealing with objects on the image in the augmented reality environment. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate overlaying call button taught by Bjontegard into modified invention off Wan such that when dealing with data in the augmented reality environment, system will be able to provide phone call button and allow user to make phone call to deal with ticket and/or seating information of the ticket they purchases which provide more user friendly user interface when using the system.
Regarding Claim 19, it recites limitations similar in scope to the limitations of Claim 9 and therefore is rejected under the same rationale.
Response to Arguments
Applicant’s arguments with respect to claim 1, filed on 4/6/2026, with respect to rejection under 35 USC § 103 have been considered but are moot in view of the new ground(s) of rejection. it has now been taught by the combination of prior arts Wan, Nurmi and Mayerle.
In regard to Claims 2-10, 12-17, 19-20, they directly/indirectly depends on independent Claim 1, 11 respectively. Applicant does not argue anything other than the independent Claim 1. The limitations in those claims in conjunction with combination previously established as explained.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20160012465 A1 System and method for distributing, receiving, and using funds or credits and apparatus thereof
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUJANG TSWEI whose telephone number is (571)272-6669. The examiner can normally be reached 8:30am-5:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YuJang Tswei/Primary Examiner, Art Unit 2614
Prosecution Insights