DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-3 and 16-32 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter.
Claims 1-3 describe a system including, “one article of apparel; one graphical image of a water display including one or more water display elements configured with the at least one article of apparel; one AR marker; an image recognition application; wherein when the image recognition application recognizes the at least one AR marker, the system augments the at least one graphical image”.
Further, Applicant’s specification, at paragraph 0017, discloses, “As another example of a plan drawing or other type of drawing depicting components along with specific patterns or markers, a viewing device with the requisite software may be aimed at this plan drawing, and an associated computer may generate a 3-D animation of the fountain jets actually spraying water. The system may then superimpose this image to generate the composite augmented image. With appropriate software, the computer also can ascertain from the pattern the angle from which the plan is being viewed and modify the 3D animation as though it were being seen from that same angle. In this way, when a user may move his/her viewing device over and around the plan drawing of the fountain, the viewer may see the fountain presented in 3D, with proper distance and perspective, as though he/she were flying over the fountain.”.
Thus, while the claim language discloses a system, said language fails to disclose that said system includes one or more hardware components and is not merely software. Thus, as evidenced by Applicant’s specification at pgs. 2-3, paragraph 0017, it appears that said claims, taken as a whole, read on computer listings per se.
Computer programs claimed as computer listings per se, i.e., the descriptions or expressions of the programs, are not physical "things." They are neither computer components nor statutory processes, as they are not "acts" being performed. Such claimed computer programs do not define any structural and functional interrelationships between the computer program and other claimed elements of a computer which permit the computer program's functionality to be realized. In contrast, a claimed non-transitory computer-readable medium encoded with a computer program is a computer element which defines structural and functional interrelationships between the computer program and the rest of the computer which permit the computer program's functionality to be realized, and is thus statutory. See Lowry, 32 F.3d at 1583-84, 32 USPQ2d at 1035.
Claims 2-3 are rejected for depending from claim 1. Additionally, claims 2-3 also fail to disclose that said system includes one or more hardware components and is not merely software.
Claims 16-32 describe a system including, “An augmented reality (AR) system for use with a display at a location, comprising: a real world object; a graphical image that is attachable to or formed as part of the real world object, and that includes at least a portion of the display; an AR viewing application that is adapted to recognize the real world object, and that determines a proximity of the real world object with respect to the location based on the recognition; wherein, when the proximity is within a threshold distance from the location, the AR viewing application augments a virtual reproduction of the graphical image on the visual display by applying at least one augmentation feature to the virtual reproduction of the graphical image”.
Further, Applicant’s specification, at paragraph 0017, discloses, “As another example of a plan drawing or other type of drawing depicting components along with specific patterns or markers, a viewing device with the requisite software may be aimed at this plan drawing, and an associated computer may generate a 3-D animation of the fountain jets actually spraying water. The system may then superimpose this image to generate the composite augmented image. With appropriate software, the computer also can ascertain from the pattern the angle from which the plan is being viewed and modify the 3D animation as though it were being seen from that same angle. In this way, when a user may move his/her viewing device over and around the plan drawing of the fountain, the viewer may see the fountain presented in 3D, with proper distance and perspective, as though he/she were flying over the fountain.”.
Thus, while the claim language discloses a system, said language fails to disclose that said system includes one or more hardware components and is not merely software. Thus, as evidenced by Applicant’s specification at pgs. 2-3, paragraph 0017, it appears that said claims, taken as a whole, read on computer listings per se.
Computer programs claimed as computer listings per se, i.e., the descriptions or expressions of the programs, are not physical "things." They are neither computer components nor statutory processes, as they are not "acts" being performed. Such claimed computer programs do not define any structural and functional interrelationships between the computer program and other claimed elements of a computer which permit the computer program's functionality to be realized. In contrast, a claimed non-transitory computer-readable medium encoded with a computer program is a computer element which defines structural and functional interrelationships between the computer program and the rest of the computer which permit the computer program's functionality to be realized, and is thus statutory. See Lowry, 32 F.3d at 1583-84, 32 USPQ2d at 1035.
Claims 17-32 are rejected for depending from claim 16. Additionally, claims 2-3 also fail to disclose that said system includes one or more hardware components and is not merely software.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 are rejected under 35 U.S.C. 103 as being unpatentable over Libin (US 12045942 B1) in view of Almeda (US 20150096108 A1).
Regarding claim 1, Libin discloses an augmented reality (AR) system (Libin, col. 2, l. 31, “the system described herein, augmenting a presentation … presenting the augmented … on computer screens”)comprising:
at least one article of apparel (Libin, col. 4, l. 65, “a t-shirt (article of apparel) designed with a constrained palette and encoding a color cycling request”);
at least one AR marker configured with the at least one graphical image (Libin, col. 4, l. 65, “a t-shirt designed with a constrained palette (pattern/AR marker configured with the at least one graphical image) and encoding a color cycling request”); and
an image recognition application adapted to recognize the at least one AR marker (Libin, col. 3, l. 14, “software that augments a presentation having a presenter wearing clothing having specific visual characteristics”; col. 5, l. 1, “The system recognizes the t-shirt, uses an enhanced segmentation technology to identify the chest area of the presenter, makes a bounding box”); and
wherein when the image recognition application recognizes the at least one AR marker, the system augments the at least one graphical image (Libin col. 5, l. 1, “The system recognizes the t-shirt, uses an enhanced segmentation technology … makes a bounding box around the chest and within the contour of the t-shirt and uses color cycling techniques to animate the t-shirt ... Upon recognizing a pattern, décor or other characteristic features of the clothing of the presenter, the system may associate additional extraneous augmented objects with the presentation”).
Libin does not disclose
at least one graphical image of a water display including one or more water display elements configured with the at least one article of apparel;
However, Almeda discloses
at least one graphical image of a water display including one or more water display elements configured with the at least one article of apparel (Almeda [0072], “The garment may include, without limitation, shirts, polos … having a specific themed design … The themed design may include, without limitation, a sea creature theme (at least one graphical image of a water display including one or more water display elements configured with the at least one article of apparel)”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Libin further with Almeda to include a water based design on the clothing. This would have enhanced Libin by including an additional artificial reality option for users to experience.
Regarding claim 2, Libin in view of Almeda discloses the system of claim 1 wherein the at least one AR marker includes at least one of the one or more water display elements (Almeda fig. 2; [0072], “Figs 1-2 illustrates a top view of an exemplary interactive garment having a permanent indicia and a machine-readable code integrated into the indicia … The themed design may include … a sea creature theme”).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Libin in view of Almeda and further view of Challagolla et al (US 20200005540 A1).
Regarding claim 3, Libin in view of Almeda discloses the system of claim 2, further comprising at least one virtual information associated with the at least one AR marker (Libin col. 5, l. 1, “The system recognizes the t-shirt, uses an enhanced segmentation technology … makes a bounding box around the chest and within the contour of the t-shirt and uses color cycling techniques to animate the t-shirt ... Upon recognizing a pattern, décor or other characteristic features of the clothing of the presenter, the system may associate additional extraneous augmented objects with the presentation”);
Libin in view of Almeda does not disclose does not disclose
wherein the at least one virtual information includes at least one virtual image of one or more water streams and wherein the system augments the at least one graphical image such that the at least one virtual image of one or more water streams appears to emanate from the at least one of the one or more water display elements.
However, Challagolla discloses
wherein the at least one virtual information includes at least one virtual image of one or more water streams and wherein the system augments the at least one graphical image such that the at least one virtual image of one or more water streams appears to emanate from the at least one of the one or more water display elements (Challagolla fig. 3; [0054], “the display device 100 is being used to provide event information 120a, 120b, 120c, which in the depicted example comprises virtual object information representative of water sprinkling (a virtual representation of a water stream emanating) from virtual sprinklers 102a, 102b, 102c. (from virtual representation(s) of the at least one water feature)”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Libin further with Challagolla to display a water animation from a corresponding related virtual representation. This would have been done to generate more realistic virtual representations of real world objects.
Claims 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Libin (US 12045942 B1) in view of Kaino et al (US 20160133057 A1).
Regarding claim 16, Libin discloses an augmented reality (AR) system for use with a display at a location (Libin, col. 2, l. 31, “the system described herein, augmenting a presentation … presenting the augmented … on computer screens”), comprising:
a real world object (Libin, col. 4, l. 65, “a t-shirt (exemplary real world object) designed with a constrained palette and encoding a color cycling request”);
a graphical image that is attachable to or formed as part of the real world object, and that includes at least a portion of the display (Libin, col. 4, l. 65, “a t-shirt designed with a constrained palette and encoding a color cycling request”; col. 7, l. 19, “display the presenter 110 wearing a fitting virtual t-shirt 180 on a presentation screen 190”);
an AR viewing application that is adapted to recognize the real world object (Libin, col. 3, l. 14, “software that augments a presentation having a presenter wearing clothing having specific visual characteristics”; col. 5, l. 1, “The system recognizes the t-shirt, uses an enhanced segmentation technology to identify the chest area of the presenter, makes a bounding box”), and
the AR viewing application augments a virtual reproduction of the graphical image on the visual display by applying at least one augmentation feature to the virtual reproduction of the graphical image (Libin col. 5, l. 1, “The system recognizes the t-shirt, uses an enhanced segmentation technology … makes a bounding box around the chest and within the contour of the t-shirt and uses color cycling techniques to animate the t-shirt ... Upon recognizing a pattern, décor or other characteristic features of the clothing of the presenter, the system may associate additional extraneous augmented objects with the presentation”).
Libin does not disclose (highlighted missing portions)
an AR viewing application that determines a proximity of the real world object with respect to the location based on the recognition;
wherein, when the proximity is within a threshold distance from the location, the AR viewing application augments a virtual reproduction
However, Kaino discloses (highlighted missing portions)
an AR viewing application that determines a proximity of the real world object with respect to the location based on the recognition (Kaino [0096], “application providing the AR content”; [0102], “specified appearance condition” may refer to a condition that a distance from the marker to the present camera position is below a specified distance threshold.”);
wherein, when the proximity is within a threshold distance from the location, the AR viewing application augments a virtual reproduction (Kaino [0102], “The AR content may appear instantly in keeping with detection of the associated marker or may appear when a specified appearance condition has also been satisfied. As one example, the expression “specified appearance condition” may refer to a condition that a distance from the marker to the present camera position is below a specified distance threshold.”)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Libin with Kaino to implement a distance threshold in the augmented reality experience system. This would have been done to improve user experience by presenting augmented reality objects only when they are relevant. See, for example, Kaino [0108], “if AR content endlessly continued to be displayed regardless of the position and posture of the camera, this would conversely appear unnatural to the user.”)
Regarding claim 17, Libin in view of Kaino discloses the system of claim 16 wherein the real world object is an article of apparel (Libin, col. 4, l. 65, “a t-shirt (an article of apparel) designed with a constrained palette and encoding a color cycling request”),
wherein the graphical image is attached to the article of apparel or is formed as part of the article of apparel while the article of apparel is manufactured (Libin col. 4, l. 40, “alternatively, the appearance, properties and behavior of the virtual augmented clothing may be encoded in the original physical clothing through recognizable patterns, colors, emblems, stickers, fashions, etc. (the graphical image is attached to the article of apparel or is formed as part of the article of apparel in the original physical clothing/while the article of apparel is manufactured)”),
wherein the graphical image includes an AR marker (Libin col. 4, l. 41, “encoded in the original physical clothing through recognizable patterns (AR marker)”),
wherein the AR viewing application is adapted to recognize the at least one AR marker (Libin col. 5, l. 11, “Upon recognizing a pattern, décor or other characteristic features of the clothing of the presenter, the system may associate additional extraneous augmented objects with the presentation”)and
wherein the proximity is determined based on the recognition of the AR marker (Kaino [0114], “In FIG. 11, a point P1 is the detection position of a marker 20a and a broken line DL1 shows a boundary where the distance from the point P1 is equal to the distance threshold d.sub.th1.”).
Claims 18, 22 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Libin in view of Kaino and further view of Almeda (US 20150096108 A1) and even further view of Challagolla.
Regarding claim 18, Libin in view of Kaino discloses the system of claim 16 but does not disclose wherein the display includes a water display with at least one water feature and the at least one augmentation feature includes a virtual representation of a water stream emanating from a virtual representation of the at least one water feature.
However, Almeda discloses
wherein the display includes a water display with at least one water feature (Almeda [0072], “The garment may include, without limitation, shirts, polos … having a specific themed design … The themed design may include, without limitation, a sea creature theme (display includes exemplary water display with at least one water feature)”)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Libin further with Almeda to include a water based design on the clothing. This would have enhanced Libin by including an additional artificial reality option for users to experience.
Libin, Kaino and Almeda do not disclose
the at least one augmentation feature includes a virtual representation of a water stream emanating from a virtual representation of the at least one water feature.
However, Challagolla discloses
the at least one augmentation feature includes a virtual representation of a water stream emanating from a virtual representation of the at least one water feature (Challagolla [0054], “the display device 100 is being used to provide event information 120a, 120b, 120c, which in the depicted example comprises virtual object information representative of water sprinkling (a virtual representation of a water stream emanating) from virtual sprinklers 102a, 102b, 102c. (from virtual representation(s) of the at least one water feature)”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Libin further with Challagolla to display a water animation from a corresponding related virtual representation. This would have been done to generate more realistic virtual representations of real world objects.
Regarding claim 22, Libin in view of Kaino and further view of Almeda and even further view of Challagolla disclose the system of claim 18 wherein the AR viewing application determines a first perspective of the AR viewing application with respect to the real world object and displays the virtual representation of the water stream emanating from the virtual representation of the at least one water feature on the visual display from the first perspective (Challagolla [0054], “the display device 100 is being used to provide event information 120a, 120b, 120c, which in the depicted example comprises virtual object information representative of water sprinkling (displays the virtual representation of the water stream emanating from the virtual representation of the at least one water feature on the visual display from a first perspective) from virtual sprinklers 102a, 102b, 102c.”).
Regarding claim 28, Libin in view of Kaino and further view of Almeda and even further view of Challagolla disclose the system of claim 18 further comprising:
an AR marker included in the graphical image (Libin col. 4, l. 41, “encoded in the original physical clothing through recognizable patterns (AR marker)”); and
a graphical user interface (GUI) adapted to edit a virtual representation of the graphical image and to virtually configure a virtual representation of the AR marker with the virtual representation of the graphical image (Libin col. 4, l . 38, “Augmented clothing may be completely controlled by a presenter”; col. 5, l. 11, “Upon recognizing a pattern, décor or other characteristic features of the clothing of the presenter, the system may associate additional extraneous augmented objects with the presentation, where some of the associated objects may be static and other objects may be dynamic and may manifest assigned characters, follow predefined behaviors, and implement various scenarios, enhancing the original presentation and partially or fully controlled by the presenter.”; col .6, l. 66, “a user interface (a graphical user interface (GUI) adapted to edit a virtual representation of the graphical image and to virtually configure a virtual representation of the AR marker with the virtual representation of the graphical image) for pre-recorded presentations and video conferences with augmented clothing”).
Claims 26 are rejected under 35 U.S.C. 103 as being unpatentable over Libin in view of Kaino and further view of Almeda and further view of Steube et al (US 20150235267 A1).
Regarding claim 26, Libin in view of Kaino and further view of Almeda and even further view of Challagolla disclose the system of claim 18 but does not disclose wherein the AR viewing application is adapted to sense a real time ambient lighting condition of the real world object, and the at least one augmentation feature includes a virtual lighting condition applied to the virtual representation of the water stream emanating from the virtual representation of the at least one water feature based at least in part on the sensed real time ambient lighting condition.
However, Steube discloses
the AR viewing application is adapted to sense a real time ambient lighting condition of the real world object, and the at least one augmentation feature includes a virtual lighting condition applied to the virtual representation of the water stream emanating from the virtual representation of the at least one water feature based at least in part on the sensed real time ambient lighting condition (Steube [0020], “The augmented reality server may receive the ambient environment information (sense a real time ambient lighting condition of the real world object), such as an image captured via the camera … The augmented reality server may identify a trigger in the received ambient environment information, for example the received image. The image may include … waves at a beach.”; [0052], “the augmented reality application 122 may receive the response action from the augmented reality server 150 and, in response, may overlay a virtual image 342 of a three-dimensional jet ski 340 on the realtime image of the waves 316”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Libin further with Steube to utilize ambient light information from a user’s environment. This would have been done to generate a more realistic augmented reality viewing experience.
Allowable Subject Matter
Claims 19-21, 23-25, 27 and 29-32 would be allowable if rewritten to overcome the rejections under 35 U.S.C. 101, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 19, none of the prior art of record disclose the claim as recited.
Claims 20-21, 24-25 and 27 are allowed for depending from claim 19.
Regarding claim 23, none of the prior art of record disclose the claim as recited.
Regarding claim 29, none of the prior art of record disclose the claim as recited.
Claim 30 is allowed for depending from claim 29.
Regarding claim 31, none of the prior art of record disclose the claim as recited.
Claim 32 is allowed for depending from claim 31.
Conclusion
See the notice of references cited (PTO-892) for prior art made of record, including art that is not relied upon but considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JITESH PATEL whose telephone number is (571)270-3313. The examiner can normally be reached 8am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said A. Broome can be reached at (571) 272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JITESH PATEL/Primary Examiner, Art Unit 2612
Prosecution Insights