Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to application 18/773,678 filed on July 16, 2024.
Claims 1-20 are pending.
Information Disclosure Statement
As required by M.P.E.P. 609(C), the applicant’s submission of the Information Disclosure Statement dated July 16, 2024 is acknowledged by the examiner and the cited references have been considered in the examination of the claims now pending. As required by M.P.E.P 609, a copy of the PTOL-1449 initialed and dated by the examiner is attached to the office action.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 8-14, and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Hill et al. (US20160314621A1) in further view of Mullins (US20150187137A1).
Regarding claim 1, Hill teaches a computer-implemented method for managing anchors in an … augmented reality system (Technologies described herein provide a mixed environment display of attached data)([0005]), the computer-implemented method comprising:
receiving, from a machine in an … environment … an identification of one or more anchors associated with the machine, a position of the machine in the industrial environment, and an orientation of the machine (The first computing device can be configured to receive status data indicating a status associated with the second computing device ... the status data may include ... location data, or any other information related to the second computing device ... the status data can be communicated directly from the second computing device ... FIG. 4B illustrates an example rendering of a display of attached data ... the visual indicator 490 is configured to show an association between the status data and the robot 102 ... a direction and velocity are described)([0005] and [0073]; a machine (i.e., second device such as a robot) transmits information including anchors (i.e., information related to the robot), location of the robot, and orientation (i.e., direction) of the robot);
creating, by the industrial augmented reality system, a augmented reality framework for the … environment, wherein the augmented reality framework includes a location of the one or more anchors associated with the machine in the … environment (The configurations described herein provide both a “see through display” and an “augmented reality display … FIG. 4B illustrates an example rendering of a display of attached data ... the visual indicator 490 is configured to show an association between the status data and the robot 102 ... the size, position, and other characteristics of the visual indicator 490 may be based on one or more factors ... graphical augmentation can be positioned in relation to the position of the real-world view of a real-world object, such as the robot 102)([0038], [0073], and [0075]; Figure 4B – a generated an augmented reality which displays information at a location (i.e., a position in relation to a robot) is shown);
receiving, by the industrial augmented reality system from the machine, an indication that one or more of the position of the machine in the industrial environment and the orientation of the machine has changed (the robot 102 may be equipped with a wheel position sensor. Data or a signal generated ... may be used to identify the location, velocity or other information regarding the robot 102)([0047]; real-time sensor information of the robot is continually transmitted, the information includes changes in location information as the robot continually moves); and
updating, the augmented reality framework for the … environment based on the indicated change to of the one or more of the position of the machine in the … environment and the orientation of the machine (The position of the graphical augmentation can move with the movement of the real-world view of the object to provide the appearance that the graphical augmentation is affixed or substantially affixed to the real-world object)([0075]; the augmented reality is updated based on location changes).
Hill differs from the claim in that Hill fails to teach managing anchors (i.e., the display of contextual information) in an industrial augmented reality system by receiving information including a machine identifier from the machine in an industrial environment.
However, managing display of contextual information an industrial augmented reality system by receiving information including a machine identifier from the machine in an industrial environment is taught by Mullins (A system and method for discovering a machine using an augmented reality application ... A default virtual user interface may be associated with the machine. The machine broadcasts a status of the machine, the default virtual user interface associated with the machine, and tracking data related to the machine … A user 102 may utilize the viewing device 101 to capture a view of a machine (e.g., factory machine) ... in a local real world environment such as at a factory … augmented reality application may provide the user 102 with an experience triggered by a physical object having a network presence (e.g., factory machine 114, 115) ... identifies tracking data related to the factory machine ... e.g., first floor in unit A of the factory, facing west ... to display information about the factory machine 114 ... by rendering a default virtual user interface associated with the factory machine … the default virtual user interface may include factory name, machine name)([0019], [0027], [0029], and [0041]; information including a machine name is received from a machine in generating contextual information of the machine in an industrial environment).
The examiner notes Hill and Mullins teach a presenting information in augmented reality. As such, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Hill to include the managing of Mullins such that anchors in an industrial augmented reality system are managed by receiving information including a machine identifier from a machine in an industrial environment. One would be motivated to make such a combination to provide the advantage of presenting relevant information to users in different environments.
Regarding claim 2, Hill-Mullins teach the computer-implemented method of claim 1, wherein the indication that one or more of the position of the machine in the industrial environment and the orientation of the machine has changed is created by the machine based on the machine detecting a change in the one or more of the position of the machine in the industrial environment and the orientation of the machine (Hill - the robot 102 may be equipped with a wheel position sensor. Data or a signal generated ... may be used to identify the location, velocity or other information regarding the robot 102)([0047]; the sensor detects a change in position of the robot).
Regarding claim 3, Hill-Mullins teach the computer-implemented method of claim 1, wherein one or more of the position of the machine in the industrial environment and the orientation of the machine are determined by the machine based on an indoor positioning system disposed in the industrial environment (Mullins - The server 110 then identifies tracking data related to the factory machine 114 as detected by ... tracking sensors 112 external to the viewing device 101 ... e.g., sensors placed in corners of a venue or a room)([0030] and [0033]).
Regarding claim 4, Hill-Mullins teach the computer-implemented method of claim 1, wherein the identification of one or more anchors associated with the machine also includes a relative location of the one or more anchors relative to the position of the machine (Hill - graphical augmentation can be positioned in relation to the position of the real-world view of a real-world object, such as the robot 102)([0075]; information is displayed at a location relative to the robot).
Regarding claim 5, Hill-Mullins teach the computer-implemented method of claim 1, further comprising receiving a first set of images, captured by an augmented reality display, of the machine and the one or more anchors in the industrial environment prior to receiving the indication that one or more of the position of the machine in the industrial environment and the orientation of the machine has changed (Hill - the augmented reality display may include an opaque display that is configured to overlay content over a display of an image, which may be from any source, such as a video feed from a camera used to display a real-world view ... As shown in FIG. 3C, a position of the robot 102 relative to an environment and an object, such as the obstacle 205, can be displayed to a user … The computing device architecture 1000 illustrated in FIG. 10 includes ... camera 1050 can be configured to capture still images and/or video)([0038], [0064], [0138], and [0170]; a camera on a user’s device continually captures images to generate an augmented reality display; Figure 3C – an exemplary augmented reality display before robot location change is shown).
Regarding claim 6, Hill-Mullins teach the computer-implemented method of claim 5, further comprising receiving a second set of images, captured by the augmented reality display, of the machine and the one or more anchors in the industrial environment after receiving the indication that one or more of the position of the machine in the industrial environment and the orientation of the machine has changed (Hill - the augmented reality display may include an opaque display that is configured to overlay content over a display of an image, which may be from any source, such as a video feed from a camera used to display a real-world view ... As shown in FIG. 3D, various status indicators attached to various stages of execution can be displayed … The computing device architecture 1000 illustrated in FIG. 10 includes ... camera 1050 can be configured to capture still images and/or video)([0038], [0065], [0138], and [0170]; a camera on a user’s device continually captures images to generate an augmented reality display; Figure 3D – an exemplary augmented reality display after robot location change is shown).
Regarding claim 8, Hill-Mullins teach the computer-implemented method of claim 1, further comprising displaying, by an augmented reality device in the industrial environment, one or more of the one or more anchors to a user based on the location and orientation of the augmented reality device (Mullins - The visualization of the default virtual user interface can be generated in relation to a location of the machine relative to the viewing device)([0022]).
Regarding system claims 9-14 and 16, the claims generally correspond to method claims 1-6 and 8, and recite similar features in system form; therefore, the claims are rejected under similar rationale.
Regarding computer program product claims 17-20, the claims generally correspond to method claims 1-4, and recite similar features in computer program product form; therefore, the claims are rejected under similar rationale.
Allowable Subject Matter
Claims 7 and 15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record on form PTO-892 and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider the reference fully when responding to this action. The document cited therein and enumerated below teaches a method and apparatus for managing presentation of information in augmented reality.
US20170270362A1
US20180130260A1
US20190147655A1
US20190262993A1
US20190389066A1
US20210042992A1
US20210094180A1
US20230001294A1
US20230162394A1
US20240112417A1
US10048753B1
US10888998B2
US10896546B2
US10950051B2
US11232307B2
US11633667B2
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Yongjia Pan whose telephone number is (571)270-1177. The examiner can normally be reached Monday - Friday, 9:00 AM - 5:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Scott Baderman can be reached at 571-272-3644. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YONGJIA PAN/Primary Examiner, Art Unit 2118
Prosecution Insights