DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 07/09/2025 has been entered.
The amendment filed of claims 1, 8, and 15 has been acknowledged.
The cancelation of claims 2, 9, and 16 has been acknowledged.
Response to Arguments
Applicant's arguments filed 07/09/2025 have been fully considered with respect to the pending claims, have been fully considered but are moot because the arguments the arguments rely on newly added and/or amended claim limitations. The examiner has revised the rejections to match the new claim limitations.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1, 8, and 15 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Amended claims 1, 8, and 15, recite “set via a supervised neural-network classifier trained on a labelled event-stream data set without frame accumulation.”; specifically, the phrase “without frame accumulation”. The specification does not describe what does and does not constitute frame accumulation and does not seem to describe a way to train on an event-stream data without it.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1, 3-4, 8, 10-11, 15, 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Everding et al. (L. Everding, L. Walger, V. S. Ghaderi and J. Conradt, "A mobility device for the blind with improved vertical resolution using dynamic vision sensors," 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom), Munich, Germany, 2016, pp. 1-5, doi: 10.1109/HealthCom.2016.7749459, hereinafter “Everding”) in view of Moreno-Rodríguez et al. (Moreno-Rodríguez, F.J., Traver, V.J., Barranco, F., Dimiccoli, M., Pla, F. (2022). Visual Event-Based Egocentric Human Action Recognition. In: Pinho, A.J., Georgieva, P., Teixeira, L.F., Sánchez, J.A. (eds) Pattern Recognition and Image Analysis. IbPRIA 2022. Lecture Notes in Computer Science, vol 13256. Springer, Cham. https://doi.org/10.1007/978-3-031-04881-4_32, hereinafter “Moreno-Rodríguez”)
Regarding claim 1, Everding teaches: A computer apparatus in a wearable device comprising: (Fig. 2)
at least one event camera, (Introduction, "OIWOB is a system that resembles the one suggested here most closely. It uses two regular cameras to capture visual information as well as an inertial measurement unit (IMU) to track head position and conveys position of obstacles and information about static background features using different sound patterns.")
at least one of the event cameras disposed to be outwardly facing; (Fig. 2)
and at least one processor in data communication with the at least one event camera and a memory storing processor executable code for configuring the at least one processor to: receive data streams from the at least one event camera, (II. Device A. Concept of event-based vision, "Their synchronized vision stream is transmitted to a computing stick on which the depth extraction is performed and events are translated into virtual spatial sounds. The sound signal is sent to the users ear via headphones connected to a USB sound adapter.")
each data stream comprising one or more pixel change events; associate the one or more pixel change events to an environmental event; and produce a determination about the environmental event based on a known data set. (II. Device A. Concept of event-based vision, "In a static setting the data stream will only convey information about moving objects; while, when the DVS is moved, most of the events will be generated at object edges or sharp textures changes. In order to be displayable, a time interval of the event stream has to accumulated to form a frame (Fig. 1)."; Examiner’s Note – Applicant argues that association with pixel change with environmental event or determination is not taught in prior art; however, in the prior art the pixel changes as seen in Fig. 1 are associated with some movement of either the robot or some other object in the environment and that association is its own determination that somethings has changed in some way.)
Everding does not teach:
However, Moreno-Rodríguez teaches set via a supervised neural-network classifier trained on a labelled event-stream data set without frame accumulation. (Moreno-Rodríguez, 1 Introduction Overview and Contributions, "We propose the use of event frames [12, 24] and a hybrid neural architecture (convolutional and recurrent neural networks) by leveraging authors’ recent related work on egocentric gesture recognition using conventional (not event-based) eyewear cameras [17, 18].").
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify a wearable device that constructs environments for navigation to include with the use of neural networks because such a modification is the result of combining prior art elements according to known methods to yield predictable results. More specifically, the wearable device’s environment construction as modified by neural networks can yield a predictable result of creating more accurate environmental reconstruction. Thus, a person of ordinary skill would have appreciated including in a wearable device that constructs environments for navigation to include with the use of neural networks since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 3, Everding in view of Moreno-Rodríguez teaches: The computer apparatus of Claim 1, wherein the at least one processor is configured to: perform spatial encoding of the data streams; and perform recurrent encoding of the data stream. (Moreno-Rodríguez, Fig. 2; Examiner's Note - The data streams are encoded into event frames and this encoding is repeated fir the input RGB frames.)
Regarding claim 4, Everding in view of Moreno-Rodríguez teaches: The computer apparatus of Claim 3, wherein the at least one processor is configured to correlate event volumes for spatial encoding. (Moreno-Rodríguez, II. Device A. Concept of event-based vision, "In a static setting the data stream will only convey information about moving objects; while, when the DVS is moved, most of the events will be generated at object edges or sharp textures changes. In order to be displayable, a time interval of the event stream has to accumulated to form a frame (Fig. 1)."; Examiner's Note - The physical objects or volumes are encoded as part of a physical environment.)
Regarding claim 8, claim 8 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Everding further teaching on: An wearable event monitoring system (Fig. 2)
Regarding claim 10, claim 10 has been analyzed with regard to claim 3 and is rejected for the same reasons of obviousness as used above as well as in accordance with Everding further teaching on: An wearable event monitoring system (Fig. 2)
Regarding claim 11, claim 11 has been analyzed with regard to claim 4 and is rejected for the same reasons of obviousness as used above as well as in accordance with Everding further teaching on: An wearable event monitoring system (Fig. 2)
Regarding claim 15, claim 15 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Everding further teaching on: A wearable comprising: at least one event camera, at least one of the event cameras disposed to be outwardly facing (Fig. 2)
Regarding claim 17, claim 17 has been analyzed with regard to claim 3 and is rejected for the same reasons of obviousness as used above as well as in accordance with Everding further teaching on: A wearable comprising: at least one event camera, at least one of the event cameras disposed to be outwardly facing (Fig. 2)
Regarding claim 18, claim 18 has been analyzed with regard to claim 4 and is rejected for the same reasons of obviousness as used above as well as in accordance with Everding further teaching on: A wearable comprising: at least one event camera, at least one of the event cameras disposed to be outwardly facing (Fig. 2)
Claim(s) 5, 12, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Everding et al. (L. Everding, L. Walger, V. S. Ghaderi and J. Conradt, "A mobility device for the blind with improved vertical resolution using dynamic vision sensors," 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom), Munich, Germany, 2016, pp. 1-5, doi: 10.1109/HealthCom.2016.7749459, hereinafter “Everding”) in view of Plizzari et al. (C. Plizzari et al., "E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition," in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 19903-19915, doi: 10.1109/CVPR52688.2022.01931., hereinafter “Plizzari”)
Regarding claim 5, Everding does not teach: The computer apparatus of Claim 1, further comprising at least one non-image sensors, wherein:the at least one processor is further configured to: receive data streams from the at least one non-image sensors; and correlate the data streams from the non-image sensor with the data streams from the event cameras; and producing the determination about the environmental event is based on both the data streams from the non-image sensor and the data streams from the event cameras.
However, Plizzari teaches The computer apparatus of Claim 1, further comprising at least one non-image sensors, wherein:the at least one processor is further configured to: receive data streams from the at least one non-image sensors; and correlate the data streams from the non-image sensor with the data streams from the event cameras; and producing the determination about the environmental event is based on both the data streams from the non-image sensor and the data streams from the event cameras. (Plizzari, First Person Action Recognition, "While the use of optical flow has been the de-facto procedure [14– 17,19,41] in FPAR, the interest has recently shifted towards more lightweight alternatives, such as gaze [27, 59, 71], audio [9, 52, 78], depth [32], skeleton [32], and inertial measurements [41], to enable motion modeling in online settings. These, when combined with traditional modalities, produce encouraging results, but not enough to make them viable alternatives. With this work, we show that the intrinsic motion information encoded by event data makes this modality potentially more suitable than RGB.")
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify wearable environment reconstruction to include motion modeling through gaze, audio, depth, skeleton and inertial measurements because such a modification is the result of combining prior art elements according to known methods to yield predictable results. More specifically, wearable environment reconstruction to include motion modeling through gaze, audio, depth, skeleton and inertial measurements can yield a predictable result of allowing for an understanding of the person’s relative position to the environment. Thus, a person of ordinary skill would have appreciated including in wearable environment reconstruction to include motion modeling through gaze, audio, depth, skeleton and inertial measurements since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 12, claim 12 has been analyzed with regard to claim 5 and is rejected for the same reasons of obviousness as used above as well as in accordance with Everding further teaching on: An wearable event monitoring system (Fig. 2)
Regarding claim 19, claim 19 has been analyzed with regard to claim 5 and is rejected for the same reasons of obviousness as used above as well as in accordance with Everding further teaching on: A wearable comprising: at least one event camera, at least one of the event cameras disposed to be outwardly facing (Fig. 2)
Claim(s) 6-7, 13-14, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Everding et al. (L. Everding, L. Walger, V. S. Ghaderi and J. Conradt, "A mobility device for the blind with improved vertical resolution using dynamic vision sensors," 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom), Munich, Germany, 2016, pp. 1-5, doi: 10.1109/HealthCom.2016.7749459, hereinafter “Everding”) in view of Gerbauer et al. (US Patent Number 2020/0348755-A1, hereinafter “Gerbauer”)
Regarding claim 6, Everding does not teach: The computer apparatus of Claim 5, wherein:at least one of the event cameras is disposed to be inwardly facing; and the at least one processor is further configured to: receive one or more data streams from the at least one inwardly facing event camera; correlate the outward facing event camera data streams and inwardly facing event camera data streams via temporal encoding; and produce a determination about a user health state.
However Gerbauer teaches: The computer apparatus of Claim 5, wherein:at least one of the event cameras is disposed to be inwardly facing; and the at least one processor is further configured to: receive one or more data streams from the at least one inwardly facing event camera; (Gerbauer, Fig. 4 ;[0021], "A gaze tracking system includes a camera and a processor that performs gaze tracking on data received from the camera regarding light from a light source reflected off the eye of a user. In various implementations, the camera includes an event camera with a plurality of light sensors at a plurality of respective locations that, in response to a particular light sensor detecting a change in intensity of light, generates an event message indicating a particular location of the particular light sensor. An event camera may include or be referred to as a dynamic vision sensor (DVS), a silicon retina, an event-based camera, or a frame-less camera.")
correlate the outward facing event camera data streams and inwardly facing event camera data streams via temporal encoding; (Gerbauer, [0027], "In some implementations, while presenting an augmented reality (AR) experience, the HMD 120 is configured to present AR content and to enable optical see-through of the scene 105. In some implementations, while presenting a virtual reality (VR) experience, the HMD 120 is configured to present VR content and to enable video pass-through of the scene")
and produce a determination about a user health state. (Gerbauer, [0040], "In some implementations, the one or more communication buses 304 include circuitry that interconnects and controls communications between system components. In some implementations, the one or more I/O devices and sensors 306 include at least one of an inertial measurement unit (IMU), an accelerometer, a magnetometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., blood pressure monitor, heart rate monitor, blood oxygen sensor, blood glucose sensor, etc.),")
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify wearable environmental reconstruction to include inward facing sensors because such a modification is the result of combining prior art elements according to known methods to yield predictable results. More specifically, wearable environmental reconstruction as modified by inward facing sensors can yield a predictable result of allowing for greater monitoring of the user and their current condition and their potential orientation relative to their environment. Thus, a person of ordinary skill would have appreciated including wearable environmental reconstruction as modified by inward facing sensors can yield a predictable result of allowing for greater monitoring of the user and their current condition since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 7, Everding in view of Gerbauer teaches: The computer apparatus of Claim 5, wherein the at least one non-image sensor comprises a temperature sensor, accelerometer, gyro, or galvanic skin sensor. (Gerbauer, [0040], "In some implementations, the one or more communication buses 304 include circuitry that interconnects and controls communications between system components. In some implementations, the one or more I/O devices and sensors 306 include at least one of an inertial measurement unit (IMU), an accelerometer, a magnetometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., blood pressure monitor, heart rate monitor, blood oxygen sensor, blood glucose sensor, etc.),")
Regarding claim 13, claim 13 has been analyzed with regard to claim 6 and is rejected for the same reasons of obviousness as used above as well as in accordance with Everding further teaching on: An wearable event monitoring system (Fig. 2)
Regarding claim 14, claim 14 has been analyzed with regard to claim 7 and is rejected for the same reasons of obviousness as used above as well as in accordance with Everding further teaching on: An wearable event monitoring system (Fig. 2)
Regarding claim 20, claim 20 has been analyzed with regard to claim 6 and is rejected for the same reasons of obviousness as used above as well as in accordance with Everding further teaching on: A wearable comprising: at least one event camera, at least one of the event cameras disposed to be outwardly facing (Fig. 2)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jinsu Hwang whose telephone number is (703)756-1370. The examiner can normally be reached M-Th 10-8 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JINSU HWANG/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights