DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because
reference characters "106" and "108" have both been used to designate the “third edible element” (see paragraph [0017]). Since reference character "106" is used to designate the “second edible element”, it appears that there is a typographical error in paragraph [0017]. Reference characters “206 and “208” have both been used to designate the “AR character” (see paragraph [0021]). Since reference character “206” is used to designate the “character”, it appears that there is a typographical error in paragraph [0021]. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3 is/are rejected under 35 U.S.C. 103 as being unpatentable over D’Andrea et al. (US 6135844 A) in view of Ntoya et al. (US 20180322705 A1), Adato et al. (US 20190215424 A1), and Paullus et al. (US 20210133449 A1).
Regarding claim 1, D’Andrea teaches an and edible element reading system ([Column 1, Lines 64-67; Column 2, Lines 1-2] reciting “The present invention is therefore directed to a book designed to make reading more enjoyable for young readers comprising at least one illustrated page and at least one hole or opening on the page in which a piece of edible candy is placed to complete the illustration or supplement the theme of the book.”) comprising:
a book comprising ([Column 2, Lines 45-47] reciting “A book in accordance with the present invention is shown generally at 10 in FIGS. 1 and 2 and comprises one or more pages 12 bound together in book form by a binding 14.”) and one or more edible elements embedded within the pages of the book ([Columns 2, Line 67; Column 3, Lines 1-3] reciting “The hole or opening 22 is shaped so as to compliment the illustration 20 such that the reader may complete the illustration 20 by placing a piece of edible candy 24 into the hole or opening 22.”);
the one or more edible elements ([Columns 2, Line 67; Column 3, Lines 1-3] reciting “The hole or opening 22 is shaped so as to compliment the illustration 20 such that the reader may complete the illustration 20 by placing a piece of edible candy 24 into the hole or opening 22.”);
with the book ([Column 2, Lines 45-47] reciting “A book in accordance with the present invention is shown generally at 10 in FIGS. 1 and 2 and comprises one or more pages 12 bound together in book form by a binding 14.”);
the book ([Column 2, Lines 45-47] reciting “A book in accordance with the present invention is shown generally at 10 in FIGS. 1 and 2 and comprises one or more pages 12 bound together in book form by a binding 14.”)
D’Andrea fails to teach an augmented reality and edible element reading system comprising:
an electronic device comprising a camera and a display;
a book comprising one or more augmented reality (AR) markers and one or more edible elements embedded within the pages of the book;
one or more sensors located beneath the one or more edible elements;
at least one memory; and at least one processor coupled to the at least one memory, the at least one processor configured to:
launch an AR software application stored on the electronic device, wherein the AR software application is configured to interact with the book;
scan, using the camera of the electronic device, the one or more AR markers, wherein the electronic device comprises at least one of a cell phone, tablet PC, laptop, or a combination thereof;
recognize, using the AR software application, the one or more AR markers;
generate, based on the one or more AR markers, one or more AR characters wherein the one or more AR characters are overlayed over the book when the one or more AR characters are viewed from the display of the electronic device and wherein the one or more AR characters correspond to the one or more AR markers;
detect, using the one or more sensors, the presence of the one or more edible elements;
generate an animated reaction of the one or more AR characters when the one or more edible elements are removed.
Ntoya teaches an augmented reality and edible element reading system ([0007] reciting “Embodiments of the present invention include an ingestible sticker including an edible paper and an edible ink on a surface of the edible paper creating a sticker image, wherein the sticker image is configured to be recognized by an augmented or virtual reality capable device.”) comprising:
an electronic device comprising a camera and a display ([0035] reciting “…The mobile screen can have a camera 32 that can be used to recognize the ingestible sticker 10 on a designated substrate 40, which is representatively shown in the illustrated embodiment as a pear 40.”; [0085] reciting “…they may be exported into AR/VR experiences across platforms. Different templates and support for Project Tango, Samsung GearVR, Oculus Rift, Microsoft HoloLens, HTC Vive, Google Cardboard, YouTube 360, and any other technology or head-up display that comes to market may be used.”);
a book comprising one or more augmented reality (AR) markers and one or more edible elements embedded within the pages of the book; ([0023] reciting “Embodiments of the present invention include edible, ingestible stickers, such as Edible Stickers©, that are associated with an Augmented Reality (AR) or Virtual Reality (VR) experience.”);
one or more sensors ([0036] reciting “to receive sensory information from the camera 32. When the camera 32 is directed at the ingestible sticker 10, the camera will receive sensory information regarding the ingestible sticker”) located beneath the one or more edible elements ([0023] reciting “Embodiments of the present invention include edible, ingestible stickers, such as Edible Stickers©, that are associated with an Augmented Reality (AR) or Virtual Reality (VR) experience.”);
at least one memory; and at least one processor coupled to the at least one memory, the at least one processor configured to ([0021] reciting “Further, the various components of these devices may be a process or thread, running on one or more processors, in one or more computing devices, executing computer program instructions and interacting with other system components for performing the various functionalities described herein. The computer program instructions are stored in a memory which may be implemented in a computing device using a standard memory device, such as, for example, a random access memory (RAM).):
launch an AR software application stored on the electronic device, wherein the AR software application is configured to interact with the book ([0050] reciting “The AR or VR capable application associated with the ingestible sticker is downloaded and loaded (S2) onto the mobile device.”; [0023] reciting “For example, in some embodiments, the sticker is readable by a mobile application or website that uses the readable sticker to create an interactive image of the sticker on a computer monitor or mobile device screen, e.g., making the sticker appear to come to life.”);
scan, using the camera of the electronic device, the one or more AR markers, wherein the electronic device comprises at least one of a cell phone, tablet PC, laptop, or a combination thereof ([0075] reciting “The user may scan a sticker at a certain designated location to enable the AR/VR function.”; [0035] reciting “The mobile device 30 may have a touchscreen 31 for example, a touch-sensitive phone or tablet and the mobile device 30 may be running an operating system, such as Android or iOS, though, the invention is not limited thereto.”);
recognize, using the AR software application, the one or more AR markers ([0035] reciting “The mobile screen can have a camera 32 that can be used to recognize the ingestible sticker 10 on a designated substrate 40, which is representatively shown in the illustrated embodiment as a pear 40.”);
generate, based on the one or more AR markers, one or more AR characters ([0047] reciting “In some embodiments, the virtual image may include a character that may interact with the real world, for example, interacting with the food or substrate to which it is attached.”) wherein the one or more AR characters are overlayed ([0047] reciting “The application software can overlay computer-generated information (i.e., a virtual image) in real time on top of the ingestible sticker”) over the book when the one or more AR characters are viewed from the display of the electronic device and wherein the one or more AR characters correspond to the one or more AR markers ([0057] reciting “For example, a character portrayed on the ingestible sticker 10 may be created from various foods including fruit and vegetables, or may portray an animal, human, or other animate or inanimate object.”);
It would have been obvious to one with ordinary skill, before the effective filing date of
the claimed invention, to have modified the method (taught by D’Andrea) to incorporate the teachings of Ntoya to provide a strong replacement for the edible elements from D’Andrea for the edible stickers from Ntoya, while also providing more activities for children using the provided AR technology from Ntoya which can include various characters, and device compatability. Doing so would provide more educational or fun information regarding the food item as stated by Ntoya ([0023] recited).
Adato teaches one or more sensors located beneath the one or more edible elements; and detect, using the one or more sensors, the presence of the one or more edible elements ([0186] reciting “With reference to FIG. 8A and consistent with the present disclosure, a store shelf 800 may include a plurality of detection elements, e.g., detection elements 801A and 801B. In the example of FIG. 8A, detection elements 801A and 801B may comprise pressure sensors and/or other type of sensors for measuring one or more parameters (such as resistance, capacitance, or the like) based on physical contact (or lack thereof) with products, e.g., product 803A and product 803B...Some examples of detection elements may include pressure sensors and/or light detectors configured to be placed above and/or within and/or under a store shelf as described above.”);
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to have modified the method (taught by D’Andrea in view of Ntoya) to incorporate the teachings of Adato to provide a type of sensor that can have detection of certain items when they are removed with Ntoya’s stickers for example, while also using D’Andrea’s books as a replacement for Adato’s shelves. Doing so would allow the detection of elements to be more consistent ([0186] recitied from Adato) as well as to have element identification regardless of signal as stated by Adato ([0189] recited).
Paullus teaches a method that can generate an animated reaction of the one or more AR characters when the one or more edible elements are removed ([0035] reciting “FIGS. 11A-11D show how systems and methods of the inventive subject matter react to removal of a pin badge from an AR device's camera's field of view.”; [0056] reciting “Finally, FIG. 11D shows augmented reality imagery 1102 on the AR device's display despite pin badge 1100 being completely removed from the AR device's camera's field of view. Though augmented reality imagery 1102 is depicted as an animation, any other type of augmented reality imagery discussed in this application can be handled in the same way when its associated pin badge is removed from an AR device's camera's field of view.”).
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to have modified the method (taught by D’Andrea in view of Ntoya) to incorporate the teachings of Paullus to provide a method that extends on the AR characters provided from Ntoya to have unique interactions like reacting to various objects being removed as taught by Paullus. Doing so would bring the system to life as well as to provide a variety of different functions as well as stated by Paullus ([0057] recited).
Regarding claim 2, D’Andrea, Ntoya, Adato, and Pallus teach all of the limitations of claim 1, and D’Andrea teaches the augmented reality and edible element reading system of claim 1, wherein one or more edible elements comprises at least one of chocolate, gummy, hard candy, sour candy, licorice, or a combination thereof ([Column 4, Lines 4-7] reciting “The candy 24 used with book 10' is preferably wafer or disk-shaped such as chocolate or bubble gum coins, although any candy capable of traversing the space behind page 12 may be used.”).
Regarding claim 3, D’Andrea, Ntoya, Adato, and Pallus teach all of the limitations of claim 1, and D’Andrea teaches the augmented reality and edible element reading system of claim 1, further comprising one or more cut-out regions corresponding to the one or more edible elements ([Column 2, Lines 64-67; Column 3, Lines 1-3] reciting “The illustration 20 as it appears on page 12 is incomplete in that at least one feature of the illustration 20 coincides with a hole or opening 22 in the page 12. The hole or opening 22 is shaped so as to compliment the illustration 20 such that the reader may complete the illustration 20 by placing a piece of edible candy 24 into the hole or opening 22.”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNNY TRAN LE whose telephone number is (571)272-5680. The examiner can normally be reached Mon-Thu: 7:30am-5pm; Fri: 7:30am-11:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached on (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHNNY T LE/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614