Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Use of indicates a limitation is not explicitly disclosed by the reference alone.
Claim(s) 1-6, 8-14, 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Metzler (US 2018/0293801) in view of Koga (US 2015/0221134).
Claim 1
Metzler discloses a computing platform (Metzler, abstract: “An Augmented Reality (AR)-system”) comprising:
at least one processor (processing unit);
at least one non-transitory computer-readable medium (disclosed in an AR device);
program instructions stored on the at least one non-transitory computer-readable medium that, when executed by the at least one processor, cause the computing platform to (disclosed in an AR device):
receive, from a first computing device being operated by a first user at a construction site, installation information for a given Quick Response ("QR") marker that has been attached to a physical object at the construction site, wherein the installation information for the given QR marker predefines an association between (i) the QR marker and (ii) (Metzler, ¶ 27, 29: “By calibrating the temporary marker 60, so to speak, the temporary marker is “made permanent” because from then on it is not only known in appearance, but also has a determined (hence: defined) pose relative to the coordinate system….The code of the temporary marker 60 is stored in the AR-system (e.g. in a cloud computer, or in the AR-device) to make the AR-device capable of recognising (detecting and identifying) the temporary marker 60.”);
store the installation information for the given QR marker (Metzler, ¶ 22: “Exemplarily, the permanent marker is shown herein comprising a QR-code, however, any other code or characteristic pattern stored in the AR-system may be applied”);
receive, from a second computing device being operated by a second user at the construction site, an indication that the second computing device has detected the given QR marker (Metzler, ¶ 3: “Augmented Reality (AR) systems are often utilised for supporting users on a site by referenced visualisations.”); and
based on the indication that the second computing device has detected the given QR marker, access the stored installation information for the given QR marker (Metzler, ¶ 4, 27: “AR-glasses or a tablet/smart phone, may reference itself to by means of the marker. Once locked in (or in other words: “logged” in) into the coordinate system, the AR-device is able to display virtual objects with a precise spatial link to the coordinate system, i.e. to the natural environment… The code of the temporary marker 60 is stored in the AR-system (e.g. in a cloud computer, or in the AR-device) to make the AR-device capable of recognising (detecting and identifying) the temporary marker 60.”); and
use the accessed installation information for the given QR marker to cause the second computing device to present the digital content related to the physical object (Metzler, ¶ 4, 27: “AR-glasses or a tablet/smart phone, may reference itself to by means of the marker. Once locked in (or in other words: “logged” in) into the coordinate system, the AR-device is able to display virtual objects with a precise spatial link to the coordinate system, i.e. to the natural environment… The code of the temporary marker 60 is stored in the AR-system (e.g. in a cloud computer, or in the AR-device) to make the AR-device capable of recognising (detecting and identifying) the temporary marker 60.”);
Metzler does not explicitly disclose the claimed client/server architecture and does not explicitly disclose association between (i) the QR marker and (ii) digital content related to the physical object that is stored at the computing platform.
However, Koga in the same field of endeavor discloses client/server with first and second users and association between (i) the QR marker and (ii) digital content related to the physical object that is stored at the computing platform (Koga, ¶¶ 37-38: “FIG. 1 is a drawing illustrating an exemplary configuration of an information processing system 10. As illustrated by FIG. 1, the information processing system 10 may include a server 11 as an example of an information processing apparatus, and one or more terminals 12-1 through 12-n (which may be collectively referred to as a "terminal 12" or "terminals 12"). The server 11 and the terminals 12 are connected to each other via, for example, a communication network 13 so as to be able to send and receive data. The server 11 manages, for example, AR markers that are examples of reference objects, one or more AR contents registered in association with each of the AR markers, and decision criteria (e.g., characteristic information of an image) for display control of the terminal 12. An AR marker specifies, for example, content information such as an AR content and a position where the content information is to be displayed”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider storing the association server side.
One of ordinary skill in the art would have motivation to coordinate display in a central location. One of ordinary skill in the art would have had a reasonable expectation of success because Metzler considers a server side embodiment, which would need communication between devices to coordinate display, markers, content, and the like.
Claim 2
Metzler discloses wherein the given QR marker comprises a given QR marker from a strip of QR tape (Metzler, ¶ 15: “marker may be embodied as a tape or a chain of rectangular sheets.”).
Claim 3
Metzler discloses wherein the digital content related to the physical object comprises virtual content that is to be superimposed on the physical object within an augmented reality (AR) overlay (Metzler, ¶ 44: “the AR-data can be overlaid on the AR-device in a spatially referenced way with respect to the scene”).
Claim 4
Metzler discloses wherein the digital content related to the physical object comprises one or more of text, an image, or a video related to the physical object (Metzler, ¶ 21: “displaying AR-objects (virtual models, photographs, text, animations, etc.) and thereby supporting the user of the AR-device in operations such as drilling.”).
Claim 5
Metzler discloses wherein the installation information for the given QR marker further predefines a location of the QR marker (Metzler, ¶ 22: “This marker is permanent, i.e. usually fixed for a longer term in a defined pose relative to the coordinate system”).
Claim 6
Metzler discloses wherein the digital content related to the physical object comprises a digital object that includes information about the physical object (e.g. drill holes; installation locations; Metzler, ¶ 35: “From then on, the AR-device may lock itself in to the coordinate system for correctly showing the drill holes 41 and other AR elements such as electrical wire”).
Claim 8
Metzler discloses wherein the first computing device is installed with software that presents a user interface for presenting digital content related to physical objects at construction sites (implicit in a computing system capable of displaying such content; Metzler, ¶ 21: “displaying AR-objects (virtual models, photographs, text, animations, etc.) and thereby supporting the user of the AR-device in operations such as drilling.”).
Claim 9
The same teachings and rationales in claim 1 are appliable to claim 9.
Claim 10
The same teachings and rationales in claim 2 are appliable to claim 10.
Claim 11
The same teachings and rationales in claim 3 are appliable to claim 11.
Claim 12
The same teachings and rationales in claim 4 are appliable to claim 12.
Claim 13
The same teachings and rationales in claim 5 are appliable to claim 13.
Claim 14
The same teachings and rationales in claim 6 are appliable to claim 14.
Claim 16
The same teachings and rationales in claim 8 are appliable to claim 16.
Claim 17
Metzler discloses a computing environment (Metzler, abstract: “An Augmented Reality (AR)-system”)comprising:
a first computing device is installed with software that presents a first user interface for inputting installation information for Quick Response ("QR") markers (e.g. a first user installing markers);
a second computing device is installed with software that presents a second user interface for presenting digital content related to physical objects at construction sites (e.g. a subsequent user viewing the site); and
a back-end computing platform (e.g. cloud computer), wherein:
the first computing device is programmed to:
receive, from a first user at a construction site, installation information for a given QR marker that has been attached to a physical object at the construction site, wherein the installation information for the given QR marker predefines an association between (i) the QR marker and (ii) digital content related to the physical object that is stored at the computing platform (Metzler, ¶ 27, 29: “By calibrating the temporary marker 60, so to speak, the temporary marker is “made permanent” because from then on it is not only known in appearance, but also has a determined (hence: defined) pose relative to the coordinate system….The code of the temporary marker 60 is stored in the AR-system (e.g. in a cloud computer, or in the AR-device) to make the AR-device capable of recognising (detecting and identifying) the temporary marker 60.”); and
transmit, to the back-end computing platform, the installation information for the given QR marker that predefines the association between (i) the QR marker and (ii) digital content related to the physical object that is stored at the computing platform (Metzler, ¶ 4, 27: “AR-glasses or a tablet/smart phone, may reference itself to by means of the marker. Once locked in (or in other words: “logged” in) into the coordinate system, the AR-device is able to display virtual objects with a precise spatial link to the coordinate system, i.e. to the natural environment… The code of the temporary marker 60 is stored in the AR-system (e.g. in a cloud computer, or in the AR-device) to make the AR-device capable of recognising (detecting and identifying) the temporary marker 60.”);
the second computing device is programmed to:
detect the given QR marker while being operated by a second user at the construction site (Metzler, ¶ 7, 32: “visually detect and identify the permanent marker…visually detect and identify the temporary marker…the invention may be an in-door construction site.”);
transmit, to the back-end computing platform, an indication that the second computing device has detected the given QR marker (Metzler, ¶ 4, 27: “(e.g. in a cloud computer, or in the AR-device) to make the AR-device capable of recognising (detecting and identifying) the temporary marker 60.”); ;
receive, from the back-end computing platform, an instruction to present digital content related to the physical object (Metzler, ¶ 35: “From then on, the AR-device may lock itself in to the coordinate system for correctly showing the drill holes 41 and other AR elements such as electrical wire”); and
based on the instruction, present the digital content related to the physical object to the second user (e.g. drill holes; ¶ 35: “From then on, the AR-device may lock itself in to the coordinate system for correctly showing the drill holes 41 and other AR elements such as electrical wire”); and
the back-end computing platform is programmed to:
receive, from the first computing device, the installation information for the given QR marker that predefines the association between (i) the QR marker and (ii) digital content related to the physical object that is stored at the computing platform (Metzler, ¶ 4, 27: “AR-glasses or a tablet/smart phone, may reference itself to by means of the marker. Once locked in (or in other words: “logged” in) into the coordinate system, the AR-device is able to display virtual objects with a precise spatial link to the coordinate system, i.e. to the natural environment… The code of the temporary marker 60 is stored in the AR-system (e.g. in a cloud computer, or in the AR-device) to make the AR-device capable of recognising (detecting and identifying) the temporary marker 60.”);
store the installation information for the given QR marker (“stored in the AR-system (e.g. in a cloud computer”);
receive, from the second computing device, the indication that the second computing device has detected the given QR marker (Metzler, ¶ 4, 27: “(e.g. in a cloud computer, or in the AR-device) to make the AR-device capable of recognising (detecting and identifying) the temporary marker 60.”); and
based on the indication that the second computing device has detected the given QR marker, access the stored installation information for the given QR marker (e.g. virtual content; “From then on, the AR-device may lock itself in to the coordinate system for correctly showing the drill holes 41 and other AR elements such as electrical wire, when the AR-device has only the temporary markers 61”); and
use the accessed installation information for the given QR marker as a basis for sending the instruction to the second computing device to present the digital content related to the physical object (¶ 35: “not only the permanent marker 11 but also or alternatively the temporary markers 61 may be used to reference the AR-device with respect to the coordinate system, i.e. the position and orientation (6-degrees-of-freedom) of the AR-device with respect to the reference coordinate system are determined.”).
Metzler does not explicitly disclose the claimed client/server architecture and does not explicitly disclose association between (i) the QR marker and (ii) digital content related to the physical object that is stored at the computing platform.
However, Koga in the same field of endeavor discloses client/server with first and second users and association between (i) the QR marker and (ii) digital content related to the physical object that is stored at the computing platform (Koga, ¶¶ 37-38: “FIG. 1 is a drawing illustrating an exemplary configuration of an information processing system 10. As illustrated by FIG. 1, the information processing system 10 may include a server 11 as an example of an information processing apparatus, and one or more terminals 12-1 through 12-n (which may be collectively referred to as a "terminal 12" or "terminals 12"). The server 11 and the terminals 12 are connected to each other via, for example, a communication network 13 so as to be able to send and receive data. The server 11 manages, for example, AR markers that are examples of reference objects, one or more AR contents registered in association with each of the AR markers, and decision criteria (e.g., characteristic information of an image) for display control of the terminal 12. An AR marker specifies, for example, content information such as an AR content and a position where the content information is to be displayed”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider storing the association server side.
One of ordinary skill in the art would have motivation to coordinate display in a central location. One of ordinary skill in the art would have had a reasonable expectation of success because Metzler considers a server side embodiment, which would need communication between devices to coordinate display, markers, content, and the like.
Claim 18
The same teachings and rationales in claim 3 are appliable to claim 18.
Claim 19
The same teachings and rationales in claim 4 are appliable to claim 19.
Claim 20
The same teachings and rationales in claim 5 are appliable to claim 20.
Claim(s) 7, 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Metzler (US 2018/0293801) in view of Koga (US 2015/0221134) and Katayama (US Patent 7,558,403)
Claim 7
Metzler as modified by Koga does not explicitly disclose, but Katayama discloses wherein the first computing device is installed with software that presents a user interface for inputting installation information for QR markers (Figs 6-15; Col. 1: “FIG. 6 is an example of a graphical user interface (GUI) presented by a setting unit 50;”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider an interface.
One of ordinary skill in the art would have motivation to allow users to edit the association data and other parameters. One of ordinary skill in the art would have had a reasonable expectation of success because all references consider setting and detection of markers and their associated data.
Claim 15
The same teachings and rationales in claim 7 are appliable to claim 15.
Additional Prior Art
Additional prior art relevant to Applicant’s disclosure but not relied upon:
Watanabe also considers collaborative marker detection:
PNG
media_image1.png
324
620
media_image1.png
Greyscale
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN M GRAY whose telephone number is (571)272-4582. The examiner can normally be reached on Monday through Friday, 9:00am-5:30pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached on (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN M GRAY/Primary Examiner, Art Unit 2611
Prosecution Insights