DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/13/2026 has been entered.
Claim Objections
Claim 8 is objected to under 37 CFR 1.75 as being a substantial duplicate of claim 7. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim 8 is not further limiting, dependent on claim 7, where each recite “wherein the augmented reality logic is further configured to generate one or more control signals corresponding to the one or more objects based on the control data.”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sarangdhar et al., U.S. Patent Number 11,132,840 B2, in view of Kumar et al., U.S. Patent Publication Number 2016/0140395 A1.
Regarding claim 1, Sarangdhar discloses a device (301, electronic device), comprising: a processor (303, processor); a memory (304, memory) communicatively coupled to the processor; and an augmented reality logic (308, VR management module ), configured to: receive an image (col. 10, lines 26-30, obtains a digital representation by capturing the digital representation of the real world environment in real time; input receiving unit receives a user-input as being a selection of an option to capture the digital representation in real time; col. 9, lines 30-35, digital representation is a direct representation of the real world environment; example of such direct representation include, but are not limited to, an image); determine device position data indicative of a position and an orientation of the device (col. 10, lines 56-62, location information may be obtained from one or more positioning systems; col. 19, lines 58-59, position of the VR enabled input unit, and spatial orientation of the VR enabled input unit; control information may be indicative of variation in a value of location information); identify one or more objects visible in the image (col. 11, lines 60-61, identifies the transmitting devices from the digital representation; 320, identification unit) based on the device position data (col. 12, lines 1-3, identifies the transmitting devices based on metadata associated with the digital representation); obtain control data corresponding to the one or more objects (col. 20, lines 2-5, control unit fetches a graphical representation of a control panel associated with selected transmitting device); and generate an augmented image by superimposing the control data on the image (col. 20, lines 11-17, sends the graphical representation of the control panel and a corresponding command to the rendering unit to render the graphical representation on the current view; col. 21, lines 1-3, superimposes the graphical representation of further status information on the digital representation).
However, it is noted that Sarangdhar discloses identification, but fails to disclose identification by using one or more computer vision techniques to detect pixels corresponding to the one or more objects.
Kumar discloses identification by using one or more computer vision techniques to detect pixels corresponding to the one or more objects (paragraph 0024, identify the pixels belonging to the object of interest; object of interest may be a hand, a body part; object of interest may be inanimate and can include a vehicle, a sign, a license plate; identify the pixels belonging to the object; object of interest may be detected using a computer vision algorithm for object detection and localization).
It would have been obvious to one of ordinary skill in the art to include the computer vision algorithm to detect pixels of the object as disclosed by Kumar, to accurately register and position overlays in the augmented reality environment as disclosed by Sarangdhar, to provide real time positioning and orientation for the localization of control information to overlay the realtime images.
Regarding claim 2, Sarangdhar discloses wherein the augmented reality logic is further configured to: access an object identification database (Col. 30, lines 29-30, access of an object to the user of the electronic device through the access sensor; col. 10, line 48 access the DR database); transmit an identification request indicative of the device position data to the object identification database (col. 30, lines 31-35, transmit an access sensing signal to the first electronic device); receive identification data from the object identification database in response to the identification request (col. 12, lines 1-3, identifies the transmiting devices based on metadata associated with the digital representation); and identify the one or more objects visible in the image based on the identification data (col. 12, lines 1-3, identifies the transmiting devices based on metadata associated with the digital representation ).
Claim(s) 3-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sarangdhar in view of Kumar as applied to claim 1 above, and further in view of Gao et al., 20230259594 A1.
Regarding claim 3, Sarangdhar discloses wherein the augmented reality logic is further configured to: receive biometric data (col. 29, lines 36-37, may include a biometric sensor); authenticate a user of the device (col. 31, lines 25-26, the cellular module may identify and authenticate electronic devices within a communication network); and determine a user identifier corresponding to the user (col. 29, lines 40-41, may recognize vital information about the user).
It is noted that while Sanrangdhar discloses a biometric sensor and further discloses authentication of electronic device, however, Sarangdhar in view of Kumar fail to disclose the biometric sensor performs authentication.
Gao discloses receive biometric data (paragraph 0081, receive biometric authentication information from the user device); authenticate a user of the device (paragraph 0081, can indicate that the user was authenticated by the user device); and determine a user identifier corresponding to the user (paragraph 0081, biometric authentication information can also include an identifier that is use the authenticate the veracity of the biometric authentication information ).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the biometric sensor as an authentication of the electronic device, to provide a measurement of a physical property or sense an operation state of the electronic device to an electronic signal for the authentication.
Regarding claim 4, Sarangdhar 11132840 col. 14, lines 15-20, master transmitting device; master transmitting device may include the location identification module in addition to the transmitting device; may obtain the real-time location from other transmitting device and then provide the real-time location to the status and location unit; col. 14, line 39, may access the SLI database.
However, Sarangdhar fails to disclose wherein the augmented reality logic is further configured to: access an administrative database; transmit an access control request indicative of the user identifier to the administrative database; and receive access control data from the administrative database in response to the access control request.
Gao discloses wherein the augmented reality logic is further configured to: access an administrative database (paragraph 0062, check if the person claimed is the person in an enrolled database of authorized users; biometric authentication has many applications, such as for performing access control to a device, system, place, or other accessible item ); transmit an access control request indicative of the user identifier to the administrative database (paragraph 0123, system controller generates a list of authentication data and transmits that data to the access control device); and receive access control data from the administrative database in response to the access control request (paragraph 0124, receiving indication (e.g., by the system controller); access control device may broadcast a beacon with information from a communication device of the control system, receive a response; paragraph 0124, transmits a result of the authentication to the access control device, which performs the function associated with the access control device based on the authentication result; for example, the access control device is associated with a door, the access control device will unlock the door if the user is authenticated).
It would have been obvious to include in the master transmitting device and access control as disclosed by Sarangdhar, the administrative and authentication as disclosed by Gao, to provide access to control over detected objects based on authentication of users.
Regarding claim 5, Sarangdhar discloses wherein the augmented reality logic is further configured to: identify one or more controllers associated with the one or more objects (col. 27, lines 34-36, user-input ids indication of control information corresponding to selected transmitting device(s) ); transmit one or more status requests to the one or more controllers ( col. 27, lines 36-39, upon receiving the control information, the control unit transmits the control information to the selected transmitting device(s) to control their operation); and receive the control data from the one or more controllers in response to the one or more status requests (col. 27, lines 40-43, status and location unit obtains updated status information and/or location information of the selected transmitting device(s) in accordance with the control information ).
Gao further discloses paragraph 0076, once the user is detected; inform the system controller that a person is within a range for accessing the function; Paragraph 0083, may be applied to any controlled function such as an authentication system used to grant access to a network function; paragraph 0092, determine an authentication status of the user at the user device and transmit the results to the control system.
Regarding claim 6, discloses wherein the identification data is indicative of one or more of: three-dimensional positional coordinates corresponding to the one or more objects (TABLE 1); object identifiers corresponding to the one or more objects (TABLE 2, Device Identifier) ; or controllers associated with the one or more objects (FIG. 5B and FIG. 5C).
Regarding claim 7 and 8, discloses wherein the augmented reality logic is further configured to generate one or more control signals corresponding to the one or more objects based on the control data (col. 20, lines 2-4, control unit fetches a graphical representation of a control panel associated with selected transmitting device).
Regarding claim 9, Sarangdhar discloses wherein the augmented reality logic is further configured to: receive an input from the user (col. 19, lines 66-67, receives a first user-input); and generate the one or more control signals based on the control data and the input (col. 20, lines 2-5, upon receiving the first user-input, the control unit fetches a graphical representation of a control panel ).
Regarding claim 10, Sarangdhar discloses wherein the augmented reality logic is further configured to transmit the one or more control signals to the one or more objects (col. 20, lines 44-45, transmits the control information to the selected transmitting device in the real world environment).
Regarding claim 11, Sarangdhar discloses wherein the device position data includes: three-dimensional positional coordinates indicative of a position of the device (TABLE 1); and three-dimensional angular coordinates indicative of an orientation of the device (col. 11, lines 1, angle of arrival).
Gao further discloses col. 18, lines 49-53, location and orientation component can also include orientation sensing components that measure the wearable appliance’s current orientation in terms of the direction of the appliance’s line of sight, the angle of the appliance relative to horizontal, etc..
Regarding claim 12, Sarangdhar discloses wherein the augmented reality logic is further configured to receive the three-dimensional angular coordinates from an Inertial Measurement Unit (IMU) (col. 10, lines 62-66, indoor position system (IPS); IPS may be based on various technologies; such technologies include, but are not limited to, magnetic positioning, inertial measurements).
Gao discloses col. 18, lines 55-60, determining a wearer’s current location and orientation, including but not limited to inertial measurement units (IMUs).
Regarding claim 13, Sarangdhar discloses wherein the augmented reality logic is further configured to determine the three-dimensional positional coordinates based on one or more Radio Frequency (RF) signals received by the device (col. 30, lines 36-38, access sensor may be a space recognition sensor such as an IR sensor, an ultrasonic sensor, a radio frequency (RF) sensor, or a radar).
Regarding claim 15, Sarangdhar discloses wherein the one or more objects are one or more electronic devices (col. 8, lines 49-54, one or more transmitting devices (TDs) 316-1, 316-2, … 316-N; operating in a real world environment ).
Regarding claims 16-18, they are rejected based upon similar rational as above claims 1, 10 and 1. Sarangdhar further discloses one or more electronic devices (col. 8, lines 49-54, one or more transmitting devices (TDs) 316-1, 316-2, … 316-N; operating in a real world environment)).
Regarding claims 19 and 20, they are rejected based upon similar rational as above claims 1 and 5. Sarangdhar further discloses a method (col. 2, lines 54-55).
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Brun et al., U.S. Patent Number 11,374,808 B2
Brun discloses a device (110, computing device), comprising: a processor (122, processor); a memory (124, memory) communicatively coupled to the processor (FIG. 9); and an augmented reality logic, configured to: receive an image (col. 16, line 26-27, receive images); determine device position data indicative of a position and an orientation of the device (col. 21, lines 62-65, image may be captured by the camera of the machine vision system and may include data indicating a position and orientation of the machine vision system when the image was captured); identify one or more objects visible in the image by using one or more computer vision techniques (col. 22, lines 29-34, in response to receiving the image; identify any asset identifiers and network assets that are recognized in the image; identification of asset identifiers and network assets may be made using a computer vision-based object recognitions algorithm) to detect pixels corresponding to the one or more objects based on the device position data (col. 25, lines 8-11, identifying an area of the image in which the asset identifier is present; processing just those pixels comprising the asset identifier area); obtain control data corresponding to the one or more objects ( col. 21, lines 46-50, may display information to the user indicating one or more physical locations of the data network where work is to be done); and generate an augmented image by superimposing the control data on the image (col. 17, lines 15-20, machine vision system may enhance the video with computer-generated graphics that provide information to the user, such as indicators which identify network assets and their connectivity, e.g., what ports the connectors of a patch cord are to be connected to and instructing or otherwise guiding the user to make those connections; col. 26, lines 26-29, asset identifier value may be displayed on the headset while the connector is in the field of view, overlaid on the image in real time, or overlaid on a context image in an augmented reality environment; col. 27, lines 18-21, augmented reality feature may be activated in response to the machine vision system detecting an asset identifier within the camera’s field of view; col. 27, lines 50-56, may also add additional virtual object to the display; identifying the network assets to provide information related to the respective network assets, e.g., information indicating that the connector should be connected to the port); (Col. 24, lines 15-22, indicator may indicate an incomplete connection if the other end of the selected patch cord terminates in a correct location(e.g., correct rack or distribution point), and an incorrect connection if the other end of the patch cord terminates in an incorrect location); (Col. 11, lines 42-45, ascertaining information about the network assets 82 from the asset identifiers 84, and a database system 88 for storing and selectively accessing information; Col. 22, lines 35-36, may also transmit an interrogation signal configured to trigger a response; Col. 22, lines 40-41, the response may enable the process to positively identify what network assets are present; col. 22, lines 44-45, positively identify a network asset in the image or location of the machine vision system Col. 19, lines 35-37, ports could be identified based on their position in a panel; col. 20, lines 61-62, equipment information may be used to query a database to retrieve further equipment information; col. 21, lines 53-56, in environments where physical location data is available, such as GPS or indoor positioning systems, these coordinates may be used to identify the location of network assets; col. 17, lines 49-51, database management system may be used to access the data stored in records of the database of the database system in response to a query; col. 17, lines 16-18, provide information to the user, such as indicators which identify network assets; col. 30, lines 49-51, may store the asset equipment identification and associated search matrix location in a memory, such as the equipment database.
Zhang et al., U.S. Patent Publication Number 2024/0077934 A1
Zhang discloses a device, comprising: a processor (432 , processor); a memory communicatively coupled to the processor (434, memory), configured to: receive an image (paragraph 0035, receive and store the visual image information); determine device position data indicative of a position and an orientation of the device (paragraph 0072, position of the device may be determined by location sensors, such as a GPS unit, one or more transceivers to generate relative position coordinates, altitude sensors or barometers, and other orientation sensors; paragraph 0079, determining the current position of a portable electronic device relative to a physical environment); identify one or more objects visible in the image by using one or more computer vision techniques to detect pixels corresponding to the one or more objects based on the device position data (paragraph 0027, advanced AR technologies, such as computer vision and object tracking, may be used to produce a perceptually enriched and immersive experience; computer vison algorithms extract three-dimensional data about the physical world from the data captured; object recognition and tracking algorithms are used to detect an object; paragraph 0099, analyzes captured image to identify the object (e.g., by applying a SLAM algorithm)); obtain control data corresponding to the one or more objects (paragraph 0109, the inference engine, in some implementations, will infer a set of probable controllable features; when the product type is a “fan”, the inference engine, in some implementations, will infer a set of probable controllable features (e.g., an on-off toggle switch for power, a variable fan speed; ); and generate an augmented image by superimposing the control data on the image (paragraph 0131, presenting the slider as an overlay relative the physical environment; FIG. 8A); paragraph 0089, IMU may cooperate with a digital motion processor or programming that gathers the raw data from the components and compute a number of useful values about the position orientation; angular velocity data from the gyroscope can be integrated to obtain the position of the mobile device (in spherical coordinates).
Schmirler et al., U.S. Patent Publication Number 10,735,691
Schmirler discloses a device, comprising: a processor; a memory communicatively coupled to the processor; and an augmented reality logic, configured to: receive an image (col. 8, lines 25-26, obtain “real world “ images); determine device position data indicative of a position and an orientation of the device (col. 18, lines 38-40, determined based on location and orientation data received by VR/AR presentation system); identify one or more objects visible in the image (col. 21, line 13, identification of devices) obtain control data corresponding to the one or more objects (col. 11, lines 4-5, selected subset of virtual control panel functions may be interfaced by the user); and generate an augmented image by superimposing the control data on the image (col. 8, lines 9-12, augment this live view with superimposed operational or status data positioned on or near the view representations of relevant machines or devices, thereby yielding an augmented reality view of the environment); and further (col. 10, lines 58-67, Authentication component 306 can be configured to confirm authorization of a user to receive and interact with a virtual control panel or other virtual or augmented reality presentation. For example, authentication component 306 can be configured to cross-reference user identification information received from a wearable appliance with control privilege information defined for the identified user).
Etwaru, U.S. Patent Publication Number 2024/0061496 A1
Etwaru discloses Paragraph 0107, target device 115 can identify at least one key point in the video data by inspecting a frame buffer using computer vision; Computer vision techniques for inspecting the frame buffer and developing frame buffer intelligence can include, but are not limited to, image recognition, semantic segmentation, edge detection, pattern detection, object detection, image classification, and/or feature recognition; paragraph 0147, target device 115 can inspect the frame buffer using computer vision to identify objects (e.g., application windows, images, text) in the image data being displayed on the target device; paragraph 0148, target device 115 can identify individual objects within a window (e.g., using computer vision, using main memory inspection) and apply an interaction to a single object within a window rather than to the window as a whole based on the location of the gesture; paragraph 0051, a frame buffer may be referred to as a memory buffer containing data representing pixels in a complete video or image frame; paragraph 0143, location can be, for example, x- and y-coordinates. In one example, the location can be determined relative to other pixels in the layer. In one embodiment, the association of the location of user input and the target location can be performed by passing the location of the user input to memory. In one embodiment, the memory can be operating system memory. In one embodiment, the memory can be main memory. The location of the user input can be used to identify the target location in another layer of content.
Hoffman 20150070347 A1
Hoffman discloses device, comprising: a processor (126, data processor); a memory (130, memory) communicatively coupled to the processor; and an augmented reality logic, configured to (118, AR client; paragraph 0057, AR client may be a software package configured to run on AR device): receive an image (paragraph 0048, receive an image frame or a sequence of images); determine device position data indicative of a position and an orientation of the device (paragraph 0061, determine the orientation of AR device; paragraph 0062, may further include a positioning device configured to estimate the physical position of AR device); identify one or more objects visible in the image (paragraph 0048, recognizing particular object(s) in an image frame) by using one or more computer vision techniques to detect pixels corresponding to the one or more objects based on the device position data (paragraph 0106, uniquely identifying the tracked object; feature package may further include data for the reference image associated with the tracked object, such as data related to reference image size (e.g., in pixels) ); obtain control data corresponding to the one or more objects; and generate an augmented image by superimposing the control data on the image (paragraph 0147, GUI generated by graphics engine; graphical overlay may be superimposed on the real life image); paragraph 0146, graphical overlay may be regarded as a GUI comprising content and user-input receiving areas, which are both scaled, translated and/or rotated on the basis of the 3D pose information so that it matches the 3D pose of the object tracked on the basis of the associated image frame 724 rendered by the imaging device. This way the graphical overlay or the GUI is displayed in perspective with the tracked object in the scene. Because the GUI is rendered in perspective, in one embodiment, touch events may be transformed to coordinates in the GUI; paragraphs 0063-0064, computer-generated graphics in the three-dimensional augmented reality environment may be displayed in perspective (e.g., affixed/snapped onto) with a tracked real world object, even when the augmented reality device is moving around in the augmented reality environment, moving farther away or closer to the real world object; paragraph 0099, parameters for controlling certain interactive features of the GUI; paragraph 0102, Once the panel is fetched and the pose information is determined, an AR engine of the AR client can generate an interactive graphical user interface using the panel information and the pose information. Said interactive graphical user interface would then be displayed in perspective with the recognized real world item within the augmented reality environment (i.e., a three-dimensional augmented reality space). Visually, the GUI interface would be displayed in perspective with the object even when the object/user moves within the augmented reality environment; paragraph 0051, allow the AR client to generate a graphical overlay displayable in perspective with a tracked object; Paragraph 0052, AR client may request content from a content provider 116 and render the content into a graphical overlay using the content layout information; Paragraph 0048, If one or more objects are recognized by the object recognition system it may return an object descriptor (e.g., object identifier or "object ID") of the recognized object(s) to the AR client; paragraph 0025, computer-vision based tracker for tracking an object in said display on the basis of at least an image of the object from the digital imaging part, said being configured for: receiving an object identifier associated with an object in an image, preferably said object identifier being generated by an object recognition system.
Edwards, U.S. Patent Publication Number 2024/0078761 A1
Edwards discloses receive an image (paragraph 0054, image of environment captured); determine device position data indicative of a position and an orientation of the device (paragraph 0042, determining the relative position of the identified device; paragraph 0050, Cartesian coordinate axis system; FIG. 1A and FIG. 1B); identify one or more objects visible in the image by using one or more computer vision techniques to detect pixels corresponding to the one or more objects based on the device position data (paragraph 0048, identifying devices; paragraph 0052, may utilize an image processing algorithm to detect an object; applying an image processing algorithm to one or more images captured; may identify pixels; paragraph 0181, may identify a pixel and/or pixels corresponding to the first network device; FIG. 2); obtain control data corresponding to the one or more objects (paragraph 0048, enabling the control of devices); and generate an augmented image by superimposing the control data on the image (paragraph 0053, generates an AR environment by generating an overlay for display over an OST display; paragraph 0182; FIG. 2, 6 and 7 ).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Motilewa Good-Johnson whose telephone number is (571)272-7658. The examiner can normally be reached Monday - Friday 6am-2:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MOTILEWA . GOOD JOHNSON
Primary Examiner
Art Unit 2616
/MOTILEWA GOOD-JOHNSON/ Primary Examiner, Art Unit 2619
Prosecution Insights