DETAILED ACTION
Claims 1-20 are pending in the present application.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant's claim for foreign priority under 35 U.S.C. 119(a)-(d). The certified copy of Korea patent application number KR10-2022-0162939 filed on 11/29/2022 has been received and made of record.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 03/18/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The claims 1, 8, and 15 refers to "a reference level corresponding to agility of a user among a plurality of reference levels indicating the agility, based on physical capability information of the user" (lines 3-4). It is unclear how the "reference level" and
"plurality of reference levels" are defined or determined. The term "agility" is not
clearly explained in a technical context, and there is no clear guidance on how
"physical capability information" is used to establish these levels. The skilled
person would have difficulty understanding what specifically constitutes these
reference levels and how they are used in the device.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-8, 10-15, and 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. PGPubs 2023/0230334 to Ko et al. in view of U.S. Patent 10,832,484 to Silverstein et al., further in view of U.S 2024/0257516 to Doken.
Regarding claim 1, Ko et al. teach a wearable device comprising: a camera; a display (Figs 1-2, abstract, par 0046-0050, an eyeglass has display, camera, and so on); at least one processor including processing circuitry; and memory including one or more storage media storing instructions, wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to (Fig 2, par 0046, par 0238, a processor and memory to perform program function):
identify a reference level of user (par 0153-0154, “in operation 1310, the processor 220 may obtain location information of an external object through the sensor 210 and obtain user preference information for the external object. The user preference information for external objects may be preference information configured in advance by a user for external objects or preference information defined by the electronic device 200 based on the access frequency of a user to external objects”),
identify at least one first visual object in an environment from an image representing the environment around the wearable deice, the image obtained through the camera (par 0063-0064, “the processor 220 may obtain location information and speed information of an external object through the visual sensor 211 and the audio sensor 212 and may obtain speed information of the electronic device 200 through the acceleration sensor 213”, par 0154, “in operation 1320, the processor 220 may determine the importance of the external object, based on the location information of the external object and the user preference information for the external object. For example, when the game store of the specific brand is located within 10 m from the electronic device 200 and the user preference for the game store is greater than or equal to the reference value, the processor 220 may determine that the importance of the game store is equal to or greater than the second importance”),
determine a risk level of the at least one first visual object (par 0068-0070, “in operation 320, the processor 220 may determine the importance of an external object by using the information on an external object … According to an embodiment, the processor 220 may obtain the relative location and relative speed of an external object with respect to the electronic device 200 by using at least a part of location information of the external object, speed information of the external object, and speed information of the electronic device 200. The processor 220 may determine the importance of the external object by using at least one information of the relative location and relative speed of the external object … the importance of an external object may be classified based on a first importance and a second importance”, par 0154, “in operation 1320, the processor 220 may determine the importance of the external object, based on the location information of the external object and the user preference information for the external object. For example, when the game store of the specific brand is located within 10 m from the electronic device 200 and the user preference for the game store is greater than or equal to the reference value, the processor 220 may determine that the importance of the game store is equal to or greater than the second importance”),
identify at least one second visual object for which a risk level higher than the reference level is determined among the at least one first visual object (par 0074-0075, “in operation 350, the processor 220 may determine whether the importance of the external object equal to or greater than the first importance is equal to or greater than the second importance”, par 0153, “The user preference information for external objects may be preference information configured in advance by a user for external objects or preference information defined by the electronic device 200 based on the access frequency of a user to external objects. For example, when a user visits a game store of a specific brand more than a reference frequency, the electronic device 200 may define that the user preference for the corresponding game store is equal to or greater than the reference value”), and
display the at least one second visual object and a visual object for the at least one second visual object (par 0007, “displaying an indicator in a second area of the display to be a first size in response to the importance which is equal to or greater than the first importance and less than a second importance, and in response to the importance which is equal to or greater than the second importance, displaying the indicator in a second size larger than the first size on the display and removing at least a part of the at least one augmented reality object or the at least one virtual reality object according to priority”, par 0146-0147, “in operation 1160, when the importance of the external object is equal to or greater than the first importance and less than the second importance, the processor 220 may control the display 231 to display an indicator in the second area to be a first size”, par 0157-0158, “in operation 1350, the processor 220 may determine whether the importance of the external object which is equal to or greater than the first importance is equal to or greater than the second importance. According to an embodiment, in operation 1360, the processor 220 may control the display 231 to display an indicator in the second area to be a first size in response to the importance of the external object which is equal to or greater than the first importance and less than the second importance”).
But Ko et al. keep silent for teaching identify a reference level corresponding to a user among a plurality of reference levels, based on physical capability information of the user.
In related endeavor, Silverstein et al. teach identify a reference level corresponding to a user among a plurality of reference levels, based on physical capability information of the user (col 3:57-57 and col 4:1-4, “The risk tolerance threshold may vary user to user based on various aspects of the user's profile (e.g., height, stride length, wingspan, movement patterns, etc.). For example, the risk tolerance threshold for identifying objects as risks within an active area at distance relative to the user may increase or decrease dependent on the length of the user's stride”, col 9:66-67 and col 10:1-49, “The process 400 begins by receiving historical risk tolerance data for a first user …. the historical risk tolerance data may comprise various image data, biometric data, and audio data regarding actions of the user when responding to event data. … an adult user who typically answers the front door if the doorbell rings would prefer to be notified of the doorbell ringing. However, a child who typically does not answer the door would not want to be notified of the doorbell. Therefore, the specific risk tolerance threshold based on audio data of a doorbell ringing may be adjusted based on the user”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Ko et al. to include identify a reference level corresponding to a user among a plurality of reference levels, based on physical capability information of the user as taught by Silverstein et al. to determine a risk tolerance threshold based on user's profile to generate an alarm about environment to potential risks based on the user's own individualized risk tolerance.
But Ko et al. as modified by Silverstein et al. keep silent for teaching identify a reference level corresponding to agility of a user among a plurality of reference levels indicating the agility.
In related endeavor, Doken teaches identify a reference level corresponding to agility of a user among a plurality of reference levels indicating the agility (par 0007, “determining that the hazardous condition may occur, based on the combination, comprises determining that a database stores an indication that the combination of the identified characteristic and the object is indicative that the hazardous condition may occur. In some embodiments, the characteristic corresponds to one or more of an age of the human, and a level of distraction of the human”, par 0036, “The user profile or account may include user information input by the user, e.g., characteristics of the user, such as gender, age, height, weight, interests, or any other suitable user information, or any combination thereof, and/or user information gleaned from monitoring a condition of the user or other activities of the user, e.g., current and/or historical biometric data of the user, facial or voice characteristics of the user, historical actions or behaviors of the user, user interactions with websites or applications (e.g., social media, or any other suitable website or application, or any combination thereof) or purchase history, or any other suitable user information, or any combination thereof. In some embodiments, the user profile or account may include user information related to cognitive and/or physical capabilities or limitations, e.g., literacy, language(s) understood, hearing or vision impairments, mobility issues, etc.”, par 0046, “the SHMS may determine a level of vulnerability of the user, and may take such level of vulnerability into account when determining whether a particular scenario constitutes a potentially hazardous household condition to the user. For example, the SHMS may reference the user profile of a particular, user which may store demographic data and/or biometric data and/or any suitable user information indicative of a current state of user 102, and/or identify characteristics of the user based on real-time observations via one or more sensors. For example, if the SHMS determines that a first user is elderly (or is child) and/or physically unfit (e.g., having mobility issues, hearing or vision impairments, etc.), and/or cognitively in decline, and/or in an angry or stressed-out state, the SHMS may be more likely to determine that a particular scenario poses a larger risk to the first user as compared to a second user who is relatively young and/or physically fit and/or in a good mood, since the first user may be more distracted or disoriented or otherwise less able to avoid the potentially hazardous condition as compared to the second user”, par 0059-0063, “a user profile may be associated with a certain type of, or manner of displaying, an augmented reality scene in connection with a potentially hazardous condition. In some embodiments, the vulnerability of the user may be based on long-term characteristics (e.g., age, disability, etc.) and/or short-term characteristics (e.g., distracted, stressed, etc.), considered in combination with a detected object proximate to the user …. the SHMS may determine a type of alert, or whether to provide an alert at all, for a potentially hazardous condition based at least in part on the vulnerability indication for a particular user….the SHMS may determine that certain hazards are common in particular ages or age groups, and/or for certain users, and provide warnings accordingly. For example, the SHMS may reference historical information for the toddler indicated in column 602 and/or other toddlers in other environments, to determine common hazards for certain ages or age groups, and tailor warnings provided to such toddler (and/or male adult and female adult indicated in column 604, which may correspond to parents of the toddler) in the particular environment based on the referenced information. For example, the SHMS may determine that there is a higher likelihood for the toddler to encounter a potentially hazardous situation that is similar to potentially hazardous situations encountered by similarly aged users”, par 0066-0067, “Column 608 may specify different relationships of a potentially hazardous object with respect to a user, e.g., whether a user is in close proximity to the potentially hazardous object, whether a potentially hazardous object is within a field of view of the user, whether the object is in a hazardous situation. Column 610 may indicate whether a vulnerable user is provided with a warning in the respective situations specified in column 608; column 612 may indicate whether a non-vulnerable user is provided with a warning in the respective situations specified in column 608, and column 614 may indicate whether a warning should be provided to a non-vulnerable user with respect to the vulnerable user in the scenarios specified in column 608”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Ko et al. as modified by Silverstein et al. to include identify a reference level corresponding to agility of a user among a plurality of reference levels indicating the agility as taught by Doken to determining that the hazardous condition may occur, an augmented reality scene associated with the potentially hazardous condition associated to the object may be generated for presentation at a user device based on user’s agility.
Regarding claim 3, Ko et al. as modified by Silverstein et al. and Doken teach all the limitation of claim 1, and Doken further teaches wherein the physical capability information of the user includes at least one of eye reaction velocity of the user, body reaction velocity of the user, identifiable range in the image through the eye of the user, or range of motion of body of the user in the environment (par 0116, “The SHMS may reference a user profile associated with the identified user, and determine one or more short-term characteristics (e.g., stress level, tiredness level, etc.) and/or long-term characteristics (e.g., age, historical preferences, cognitive and/or physical capabilities or limitations such as, for example, literacy language(s) understood, hearing or vision impairments, mobility issues, etc.) to determine whether the user is a vulnerable user”).
Regarding claim 4, Ko et al. as modified by Silverstein et al. and Doken teach all the limitation of claim 1, and further teach wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to obtain information for the image, wherein the information for the image includes at least one of 2 dimension (2D) information of the image, information for depth of the environment, information for a unit configuring the environment, or 3D position information of the at least one first visual object (Ko et al.: par 0050, “The visual sensor 211 may capture still images and moving images of the surrounding environment that changes according to the movement of the electronic device 200. Through the visual sensor 211, the processor 220 may recognize the space of the surrounding environment and recognize a change in plane to measure the location of an external object and the distance to the electronic device 200 from the external object, with respect to the electronic device 200”, Silverstein et al.: col 7:32-42, “he external environment 200 includes IoT device 225A, 225B, and 225N (collectively referred to as IoT devices 225). These IoT devices 225 collect event data (e.g., image data, sound data, motion data) and send the event data to the VR device 202”, Doken: par 0035, “The field of view may comprise a pair of 2D images to create a stereoscopic view in the case of a VR device; in the case of an AR device (e.g., smart glasses), the field of view may comprise 3D or 2D images, which may include a mix of real objects and virtual objects overlaid on top of the real objects using the AR device (e.g., for smart glasses, a picture captured with a camera and content added by the smart glasses)”).
Regarding claim 5, Ko et al. as modified by Silverstein et al. and Doken teach all the limitation of claim 4, and further teach wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to obtain information for each visual object of the at least one first visual object and position information of the user, based on the information for the image, wherein the information for each visual object includes at least one of configuration information of a visual object, distance from a visual object to the user, or velocity of a visual object (Ko et al.: par 0050, “Through the visual sensor 211, the processor 220 may recognize the space of the surrounding environment and recognize a change in plane to measure the location of an external object and the distance to the electronic device 200 from the external object, with respect to the electronic device 200”, par 0064, “the processor 220 may obtain location information and speed information of an external object through the visual sensor 211 and the audio sensor 212 and may obtain speed information of the electronic device 200 through the acceleration sensor 213”, Silverstein et al.: col 3:57-67 and col 4:1-4, “the risk tolerance threshold for identifying objects as risks within an active area at distance relative to the user may increase or decrease dependent on the length of the user's stride”, Doken: par 0037-0038, “the SHMS may determine that user 102 is in proximity to one or more of objects 112, 114, 118, 120 based on comparing the current location of user 102 to the stored location of each respective object. For example, user 102 may be considered in proximity to an object if the comparison indicates that the location of user 102 and one of the objects is the same or is within a threshold distance (e.g., five feet, or any other suitable distance, or any combination thereof)”).
Regarding claim 6, Ko et al. as modified by Silverstein et al. and Doken teach all the limitation of claim 5, and further teach wherein the risk level is determined for each of the at least one first visual object based on the position information of the user and the information for each visual object (Ko et al.: par 0068-0069, “the processor 220 may obtain the relative location and relative speed of an external object with respect to the electronic device 200 by using at least a part of location information of the external object, speed information of the external object, and speed information of the electronic device 200. The processor 220 may determine the importance of the external object by using at least one information of the relative location and relative speed of the external object.”, Silverstein et al.: col 3:57-67 and col 4:1-4, “the risk tolerance threshold for identifying objects as risks within an active area at distance relative to the user may increase or decrease dependent on the length of the user's stride”, Doken: par 0037-0038, “the SHMS may determine that user 102 is in proximity to one or more of objects 112, 114, 118, 120 based on comparing the current location of user 102 to the stored location of each respective object. For example, user 102 may be considered in proximity to an object if the comparison indicates that the location of user 102 and one of the objects is the same or is within a threshold distance (e.g., five feet, or any other suitable distance, or any combination thereof)”).
Regarding claim 7, Ko et al. as modified by Silverstein et al. and Doken teach all the limitation of claim 1, and further teach wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to: in response to an average value of the risk level larger than or equal to a threshold value, identify a state of the environment as a first state, and in response to the average value of the risk level less than the threshold value, identify the state of the environment as a second state, wherein the at least one second visual object has the risk level higher than the reference level applied a weight value in response to the state of the environment is the first state (Ko et al.: par 0007, “displaying an indicator in a second area of the display to be a first size in response to the importance which is equal to or greater than the first importance and less than a second importance, and in response to the importance which is equal to or greater than the second importance, displaying the indicator in a second size larger than the first size on the display and removing at least a part of the at least one augmented reality object or the at least one virtual reality object according to priority”, par 0146-0147, “in operation 1160, when the importance of the external object is equal to or greater than the first importance and less than the second importance, the processor 220 may control the display 231 to display an indicator in the second area to be a first size”, par 0157-0158, “in operation 1350, the processor 220 may determine whether the importance of the external object which is equal to or greater than the first importance is equal to or greater than the second importance. According to an embodiment, in operation 1360, the processor 220 may control the display 231 to display an indicator in the second area to be a first size in response to the importance of the external object which is equal to or greater than the first importance and less than the second importance”, Silverstein et al.: col 4:5-12, “multiple risk tolerance thresholds may be generated depending on the type of risks and where the risks are detected. For example, risk tolerance thresholds may be generated for risks that may cause a tripping hazard within the user's active area. In addition, risk tolerance thresholds may be generated for risks that may be occurring outside the user's active area (e.g., within another room, outside of the house, etc.)”, col 6:29-46, “As more data is learned by the system 100, the weights of the neural network can be adjusted, automatically, by processor 104. Over time, the system 100 can become more accurate in determining what type of risks require an alert notification to be sent to the user when interacting with the VR simulation”, col 9:18-36, “a user may be warned via an indicator (e.g., small flashing light on UI) that the user is about to bump into an object (e.g., low risk). In another example, the VR simulation may display a live feed of a pet eating a hazardous material (e.g., high risk) on the UI. In another embodiment, the system may automatically pause the VR simulation based on the risk level (e.g., potential serious injury)”, col 10:17-38, “By using machine learning, the system may analyze the actions taken by the first user in response to the historical event data. Once the actions are analyzed, weights can be applied to the historical event data based on the actions taken by the first user … the historical event data may be categorized into critical events and non-critical events based on the weights”, Doken: par 0038, “the threshold for proximity may vary based on a type of identified object. For example, an object posing a relatively higher risk to user 102 may be associated with a greater threshold distance and an object posing a relatively lower risk to user 102 may be associated with a lower threshold distance, to give user 102 more time to react to a warning associated with the object posing a relatively higher risk to user 102”).
Regarding claims 8 and 10-14, the method claims 8 and 10-14 are similar in scope to claims 1 and 3-7 and are rejected under the same rational.
Regarding claim 15, Ko et al. teach a non-transitory computer-readable storage medium, when individually or collectively executed by at least one processor of a wearable device comprising a camera and a display, stores one or more programs including instructions that cause the wearable device to (par 0238). The remaining limitations of the claim are similar in scope to claim 1 and rejected under the same rationale.
Regarding claims 17-20, Ko et al. as modified by Silverstein et al. and Doken teach all the limitation of claim 15, the claims 17-20 are similar in scope to claims 3-5 and 7 and are rejected under the same rational.
Claim(s) 2, 9 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. PGPubs 2023/0230334 to Ko et al. in view of U.S. Patent 10,832,484 to Silverstein et al., further in view of U.S 2024/0257516 to Doken, further in view U.S. PGPubs 2018/0184958 to Publicover et al..
Regarding claim 2, Ko et al. as modified by Silverstein et al. and Doken teach all the limitation of claim 1, and keep silent for teaching wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to: display a virtual object or an interface for initial setup of the wearable device through the display, and receive a response of the user for the virtual object or the interface, wherein the physical capability information of the user is determined based on the response of the user.
In related endeavor, Publicover et al. teach wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to: display a virtual object or an interface for initial setup of the wearable device through the display, and receive a response of the user for the virtual object or the interface, wherein the physical capability information of the user is determined based on the response of the user (par 0046, “a system is provided for determining reaction times, magnitudes of responses, and/or other responses that includes a device configured to be worn on a wearer's head … The one or more processors may be configured for identifying reference times of external events and analyzing the eye-tracking images after the reference times to determine responses of the first eye, eyelid, and pupil to the external events. For example, processor(s) may identify two or more parameters selected from: a) location of at least one of the pupil and iris in one or more dimensions, b) a size of the pupil, c) a shape of the pupil, and d) a location of the eyelid from the eye-tracking images, and detect changes in the two or more parameters to determine reaction times or other responses of the wearer to external events”, par 0053, “the processor(s) may monitor changes in the orientation of the wearer's head after the reference time to determine a reaction time or other response of the wearer's head. Optionally, the processor(s) may analyze one or more of the determined reaction times to determine information regarding the wearer. For example, the reaction time of the wearer's head may be compared to the reaction time of the first eye to determine at least one of a physical or mental state of the wearer”, par 0092-0094, “Following the display of the explosion 61, measurements of the eye 62b from images acquired by the eye-tracking camera may reveal that the center of the pupil 63b has moved and/or that the pupil 63b has significantly dilated. Eye movement in a selected dimension, e.g., along a vertical or “y” axis, is depicted in an eye location time trace 67b in FIG. 6C, where the time of occurrence of initial eye movement in response to the virtual explosion is registered at time 67a. The temporal difference between the display of an explosion 61 at time 66a and significant movement of the eye at time 67a is the reaction or response time 67c of the eye …. Tracking the combination of whether a response to the display of an image causes attraction (i.e., focus) or aversion to the image along with measuring the degree of pupillary dilation may, for example, assist in the diagnosis of post traumatic stress disorder and/or to measure the effectiveness of advertising. Thus, the integration of reaction times and/or magnitudes of multiple responses by a wearer may be monitored to more accurately diagnose, predict, and/or analyze behavior of the wearer than monitoring eye movement or another parameter alone”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Ko et al. as modified by Silverstein et al. and Doken to include wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to: display a virtual object or an interface for initial setup of the wearable device through the display, and receive a response of the user for the virtual object or the interface, wherein the physical capability information of the user is determined based on the response of the user as taught by Publicover et al. to measure responses and reaction times continuously over extended periods, even over a lifetime to measure consequences of the aging process to improve performance by decreasing reaction times through training.
Regarding claim 9, Ko et al. as modified by Silverstein et al. and Doken teach all the limitation of claim 8, the claim 9 is similar in scope to claim 2 and is rejected under the same rational.
Regarding claim 16, Ko et al. as modified by Silverstein et al. and Doken teach all the limitation of claim 15, the claim 16 is similar in scope to claim 2 and is rejected under the same rational.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jin Ge whose telephone number is (571)272-5556. The examiner can normally be reached 8:00 to 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at (571)272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JIN . GE
Examiner
Art Unit 2619
/JIN GE/Primary Examiner, Art Unit 2619
Prosecution Insights