DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/18/26 has been entered.
Claim Status
Claims 1-15, 18-19, and 21 are pending. Claims 1, 19, and 21 have been amended. Claims 16-17, 20, and 22-24 have been cancelled.
Response to Arguments
Most of Applicant’s arguments (Remarks pp. 7-8) with respect to 35 USC § 103 rejections based on amended features are moot in view of the Examiner’s new ground of rejections.
In addition, Applicant states:
PNG
media_image1.png
124
564
media_image1.png
Greyscale
The Examiner disagrees.
“The edge cloud server 306 runs various applications, for example Platform-as-a-Service (PaaS) applications, and communicates with the RAN, for example using a transmission control protocol, TCP.” Westberg p. 13.
After the combination with the primary reference Qzery, edge cloud server 306 provides services that are similar to Qzery fig 1’s server 110.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-15, 18-19, and 21 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 has been amended to recite “one or more label . . . comprising a priority value.” However, the specification does not support the claimed feature.
Applicant’s disclosure does disclose “priority value,” the closest are from the following paragraphs:
[0123] In some embodiments, processing the event data further includes: assigning a priority value to the event data (e.g., either by the user device, by the edgecloud/server, or both, based on one or more of the event data, location, labels, user preferences, information about the user (e.g., characteristics, such as impaired vision or hearing)). In some embodiments, the set of criteria includes the priority value.
[0124] In some embodiments, the priority value is included in a transmission of event data to a remote processing resource (e.g., to the edgecloud) or included in a received transmission of event data (e.g., from the user device).
Here, the specification discloses that priority values are assigned to event data based on labels. The specification does not disclose that labels comprises/contains priority values. The distinction is sharpened by Applicant’s goal of the amendment to distinguish from the Examiner’s primary reference, the teaching of which is consistent with the specification.
The primary reference Ozery discloses, “For example, extremely high levels of priority notifications include environmental emergencies and device-specific emergencies. High levels of priority notifications include phone calls and task notifications. Medium levels of priority notifications include texts and emails. Low levels of priority notifications include application updates and reminders. The higher the level of priority, the more likely the smart audio system will cause a display of the notification in the display of the HMD (within a field of view of the user) in addition to causing an auditory signal in the HMD (e.g., audio notification with an adjusted volume level based on background noise).” Ozery col. 2 lines 54-67.
Independent claims 19 and 21 contains a similar limitation and have the same deficiency. These claims are also rejected. All the dependent claims are rejection because they inherit the identified deficiency.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 12-13 are rejected, and Claim 12 recites the limitation "the priority value". There is insufficient antecedent basis for this limitation in the claim. Both Claim 1 and Claim 12 have introduced “a priority value” (Claim 1 line 12; Claim 12 line 3). Claim 1 is Claim 12’s parent. Therefore, it is unclear which introduced “a priority value” Claim 12’s “the priority value” is referring to. Appropriate correction is required.
Claim 13 is rejected because Claim 13 depends on Claim 12 and inherits the identified deficiency.
Compact Prosecution
With respect to Claim Interpretation, the Examiner has provided some notes regarding “[BRI on the record]” throughout the Office Action, so that the record is clear about the scope of the claimed invention, and the record is also clear about the basis for the Examiner’s analyses. A clear record of the claim interpretation could expedite the examination by creating the condition to allow the examination to focus on Applicant’s inventive concept and its comparison with related prior art.
If there are disagreements, Applicant may present an alternative interpretation based on MPEP 2111. The Examiner will adopt Applicant’s interpretation on the record, if Applicant’s interpretation is reasonable and/or arguments are persuasive.
Applicant may amend claims relying on the Examiner’s claim interpretation provided on the record.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 7-9, 14, 18-19, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Ozery (US 10127731 B1) in view of Westberg et al. (WO 2019132744 A1) and Bucsa et al. (US 20160098305 A1).
Regarding Claim 1, Ozery teaches A method for processing of event data at a computing cloud that provides cloud computing for a wireless communication network and for providing feedback to a user device capable of extended reality (XR) presentation (
PNG
media_image2.png
534
466
media_image2.png
Greyscale
“A head mounted device (HMD) has a transparent display. The HMD detects an event external to the HMD and determines a location of the event relative to the location of the HMD. The HMD generates augmented reality (AR) content identifying the event and a direction of the location of the event relative to the location of the HMD and displays the AR content in the transparent display.” Ozery Abstract.
The claimed “feedback” includes, as an example, “(AR) content identifying the event and a direction of the location of the event.” This AR feedback is generated by processing of event data.
The disclosed “HMD” is a user device capable of extended reality (XR) presentation, because AR is one type of XR presentation.
Ozery teaches that AR feedback generated by processing the event data could be processed on a cloud server based on cloud computing, stating “The server 110 may be part of a network-based system. For example, the network-based system may be or include a cloud-based server system that provides AR content (e.g., augmented information including 3D models of virtual objects related to physical objects in images captured by the HMD 101) to the HMD 101.” Ozery col. 6 lines 40-45.
Ozery teaches a wireless communication network, stating “Accordingly, the network 108 may be a wired network, a wireless network (e.g., a mobile or cellular network), or any suitable combination thereof” (Ozery col. 9 line 64- col. 10 line 5) and “The event detection application 214 detects an event using sensors 202 (or via wireless communication from another device)” (Ozery col. 14 lines 23-39).), comprising:
receiving event data from one or more sensors as data
“The event detection application 214 detects an event using sensors 202 (or via wireless communication from another device). The event could be a truck backing up, a siren, a gunshot, an explosion, or any other type of event that the user 102 of the HMD 101 would want to know. Audio data from the sensors 202 can be used to determine the type of events using unique audio signatures or audio patterns.” Ozery col. 14 lines 23-39.
The claimed “sensory environment” includes other sensory elements disclosed in Ozery col. 7, lines 1-35; col. 11 lines 30-37; and col. 15 lines 26-41.
Note fig. 1 112 sensors are associated with fig. 1 114 Physical Environment that are further associated with fig. 1 101 Head Mounted Device.
Ozery teaches a wireless communication network, stating “Accordingly, the network 108 may be a wired network, a wireless network (e.g., a mobile or cellular network), or any suitable combination thereof” (Ozery col. 9 line 64- col. 10 line 5) and “The event detection application 214 detects an event using sensors 202 (or via wireless communication from another device)” (Ozery col. 14 lines 23-39).
Data from fig. 1 112 sensors are received through fig. 1 108 Network and fig. 1 110 Sever.
After the combination of the secondary reference, the 110 Server would become edgecloud, which as shown in fig. 1 to have received sensory data from the sensors. );
processing the event data at the computing cloud, including:
performing a localization operation using the event data to determine location data representing the event in the sensory environment (
“A head mounted device (HMD) has a transparent display. The HMD detects an event external to the HMD and determines a location of the event relative to the location of the HMD. The HMD generates augmented reality (AR) content identifying the event and a direction of the location of the event relative to the location of the HMD and displays the AR content in the transparent display.” Ozery Abstract.
Ozery provides an example, stating “In the latest sampling, the smart audio system registers the sound of a warning (e.g., a truck backing up) in a certain location. The smart audio system approximates and localizes the proximity of the user to a warning sound and notifies the user of the HMD via AR content as to where the potential warning is coming from (e.g., by giving user directional awareness, even in loud environment).” Ozery col. 2 lines 34-43.
Ozery provides a further example, stating “The array of microphones enables the HMD to pick up ambient sound and identify a location of a particular sound based on the audio data from the array of microphones. The array of microphones is thus used to detect the event and to identify the event based on the audio data.” Ozery col. 4 lines 9-17.);
performing a labeling operation using the event data to determine one or more label representing the event and comprising a priority
[BRI on the record] With respect to “label representing the event,” the Examiner reading the limitation to mean information that describes an aspect of an event, examples of which include “semantic labels,” “proximity labels,” and “magnitude/intensity labels.” This interpretation is in light of the specification:
[0071] . . . Examples of labels that can be compatible with this system:
[0072] Semantic labels: labels that project recognition context through the visual, audio, or tactile production of words communicating the type of stimulus located.
[0073] Proximity labels: labels that indicate the proximity (and (optionally) other pose information such as orientation) of particular stimuli in an end user's environment. This may include arrows indicating the direction of detected stimulus in an end user's environment along with representations (audio, visual, or tactile) of the distance or proximity of the detection.
[0074] Magnitude/intensity labels: labels that indicate the magnitude of intensity of the sensory stimuli identified. This may include an increasing or decreasing pattern of lights, sounds, or haptic feedback proportional to the magnitude or intensity of the stimulus.
Published Spec. ¶¶ 71-74.
[Mapping Analyses]
PNG
media_image3.png
402
464
media_image3.png
Greyscale
The claimed determined “label” could be mapped to fig. 13 1302 and 1304 or identification/labeling an event as worthy certain level or priority.
Ozery teaching combining a priority, stating “For example, extremely high levels of priority notifications include environmental emergencies and device-specific emergencies. High levels of priority notifications include phone calls and task notifications. Medium levels of priority notifications include texts and emails. Low levels of priority notifications include application updates and reminders. The higher the level of priority, the more likely the smart audio system will cause a display of the notification in the display of the HMD (within a field of view of the user) in addition to causing an auditory signal in the HMD (e.g., audio notification with an adjusted volume level based on background noise).” Ozery col. 2 lines 54-67.);
determining, based on a set of criteria, whether to perform a prioritization action related to the event data based on the priority
“For example, extremely high levels of priority notifications include environmental emergencies and device-specific emergencies. High levels of priority notifications include phone calls and task notifications. Medium levels of priority notifications include texts and emails. Low levels of priority notifications include application updates and reminders. The higher the level of priority, the more likely the smart audio system will cause a display of the notification in the display of the HMD (within a field of view of the user) in addition to causing an auditory signal in the HMD (e.g., audio notification with an adjusted volume level based on background noise).” Ozery col. 2 lines 54-67. “In another example embodiment, the smart audio system also defines a task hierarchy based on the priority of the task. For example, enterprise-specific tasks can be defined in the order in which the enterprise is interested in having the user/worker be interrupted. High priority tasks/actions are not to be interrupted using notifications. Low priority tasks allow for notifications interruption.” Ozery col. 3 lines 1-9.
The claimed “prioritization action” is mapped to prioritization of certain type of notification related to an event based on priority by a means that is more likely to catch a person’s attention, or prioritization of notification over a task being performed.
Ozery provides examples for relatively low and high priority events “based on a set of criteria,” stating “An example of a high priority task includes fixing a gas leak in a pipe. An example of a low priority task includes checking on a furnace filter.” Ozery col. 3 lines 7-9.);
in response to determining to perform a prioritization action, generating an output to perform one or more prioritization action (
PNG
media_image3.png
402
464
media_image3.png
Greyscale
“The AR content can be, for example, an arrow with a warning sound that plays above current environmental noise levels. The arrow will show the user something to be aware of, but will not necessarily engage the entire system or the factory in an emergency situation.” Ozery col. 9 lines 7-11.), by one or more sensory feedback device of the user device (“The HMD detects an event external to the HMD and determines a location of the event relative to the location of the HMD. The HMD generates augmented reality (AR) content identifying the event and a direction of the location of the event relative to the location of the HMD and displays the AR content in the transparent display.” Ozery Abstract.), of sensory feedback in at least one sensory dimension based on the event data, the location data, and the one or more label (
“In one example embodiment, the HMD includes a transparent display. The HMD detects an event (e.g., an explosion, a siren, a gunshot, other types of sound) external to the HMD. In another example, the event includes a receiving an electronic communication that includes a warning notification (e.g., overheated engine). The HMD determines a location of the event relative to the location of the HMD and generates AR content identifying the event and a direction of the location of the event relative to the location of the HMD. The HMD displays the AR content in the transparent display.” Ozery col.3 line 66 - Ozery col.4 line 8.
The “sensory dimension” includes visual dimension.
The “feedback” includes AR warning in the form of “label,” which includes fig. 13 1302 and 1304 or identification/labeling an event as worthy certain level or priority.); and
transmitting the output to the user device via the wireless communication network (
PNG
media_image2.png
534
466
media_image2.png
Greyscale
Ozery teaches that AR feedback generated by processing the event data could be processed on a cloud server based on cloud computing, stating “The server 110 may be part of a network-based system. For example, the network-based system may be or include a cloud-based server system that provides AR content (e.g., augmented information including 3D models of virtual objects related to physical objects in images captured by the HMD 101) to the HMD 101.” Ozery col. 6 lines 40-45. Ozery teaches the use of a wireless communication network, stating “Accordingly, the network 108 may be a wired network, a wireless network (e.g., a mobile or cellular network), or any suitable combination thereof” (Ozery col. 9 line 64- col. 10 line 5) and “The event detection application 214 detects an event using sensors 202 (or via wireless communication from another device)” (Ozery col. 14 lines 23-39).
Ozery provides an example, stating “In another example embodiment, the HMD receives a notification of the event from a server. The notification identifies a type of event, AR content identifying the type of event, and an escape route based on the type of event. The escape route illustrates a path away from the location of the event.” Ozery col. 4 lines 37-42.).
Ozery does not explicitly disclose the computing cloud is edgecloud that provides edge and cloud computing at an edge node of a communication network and data packets sent over the communication network.
Ozery does not explicitly disclose a priority is represented by a priority value.
Ozery does not explicitly disclose augmenting the event data using a third-party service.
Westberg teaches
the computing cloud is edgecloud that provides edge and cloud computing at an edge node of a communication network (
PNG
media_image4.png
410
728
media_image4.png
Greyscale
“The edge cloud server 306 runs various applications, for example Platform-as-a-Service (PaaS) applications, and communicates with the RAN, for example using a transmission control protocol, TCP.” Westberg p. 13.
The edge cloud server/node 306, an edge node, provides edge and cloud computing, because it is at the edge and part of cloud, as compared to “Centralized Cloud” 305 in fig. 3.
Both Ozery and Westberg teach communication network, among 301, 306, and 305.) and
data packets sent over the communication network (
“Although the description is given for a ‘client’ or wireless device, or user equipment (UE), it should be understood by the skilled in the art that “UE” is a non-limiting term comprising any mobile or wireless terminal, device or node equipped with a radio interface allowing for at least one of: transmitting signals in uplink (UL) and receiving and/or measuring signals in downlink (DL).” Westberg p 5.
“The step of controlling communication may comprise controlling a flow of packets sent by an application running on the edge cloud server to the radio access network during an initial period of packet transmission.” Westberg p. 8.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Westberg’s edgecloud and packet based on communication with Ozerv. One of ordinary skill in the art would be motivated to improved performance of cloud computing by strategically placing servers. “For this reason a solution called edge computing has been proposed, and it has been shown that this solution can reduce data transfer times considerably, but that the short distance alone is not enough, and as such new transport protocols must be deployed.” Westberg ¶ 6. One of ordinary skill in the art would be motivated to lower cost and enhance reliability by using established packet based communication/transmission protocols. “In an embodiment where the communication protocol is TCP, the packet start and/or packet end information comprises SYN and FIN/RST flags within a TCP packet flow.” Westberg ¶ 66.
Ozery in view of Westberg does not explicitly disclose a priority is represented by a priority value.
Ozery in view of Westberg does not explicitly disclose augmenting the event data using a third-party service.
Bucsa teaches
a priority is represented by a priority value (
“In some embodiments, a priority value is used to determine which configuration update should override other configuration updates. For example, if a first configuration update is received having a high priority and is applied at a configured device, the configured device may decide not to apply a subsequent conflicting configuration update having a lower priority.” Bucsa ¶ 94.
The Examiner takes an Official Notice that it would have been well-known in the art that a value ), and
augmenting the event data using a third-party service (
“For example, the third-party service 30 may receive raw data from a sensor such as a smoke detector, and/or processed data summarizing events that led up to an alarm condition, among other possibilities.” Bucsa ¶ 81.
The raw sensory data are augmented by summarization and analyses.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Bucsa’s priority values and augmentation of event data with Ozerv in view of Westberg. One of ordinary skill in the art would be motivated to 1) make it easier to compute with values in a computing system, and 2) to allow flexible setup of a system, e.g., security system. For example, third parties could be replaced based on cost and expertise. Third party services also lessen the computation burden and required expertise on local resources.
Regarding Claim 2, Ozery further teaches The method of claim 1,
wherein the at least one sensory dimension includes one or more of visual sensory output, audio sensory output, haptic sensory output, olfactory sensory output, and gustatory sensory output (“The AR content can be, for example, an arrow with a warning sound that plays above current environmental noise levels. The arrow will show the user something to be aware of, but will not necessarily engage the entire system or the factory in an emergency situation.” Ozery col. 9 lines 7-11.), and wherein the sensory feedback represents the location data and the one or more label to indicate presence of the event in the sensory environment (
“A head mounted device (HMD) has a transparent display. The HMD detects an event external to the HMD and determines a location of the event relative to the location of the HMD. The HMD generates augmented reality (AR) content identifying the event and a direction of the location of the event relative to the location of the HMD and displays the AR content in the transparent display.” Ozery Abstract.
The “feedback” includes AR warning in the form of “label,” which includes fig. 13 1302 and 1304 or identification/labeling an event as worthy certain level or priority.).
Regarding Claim 3, Ozery further teaches The method of claim 1, wherein the at least one sensory dimension, of the sensory feedback, differs in type from a captured sensory dimension of the one or more sensors (
Ozery discloses visual “sensory feedback,” stating “A head mounted device (HMD) has a transparent display. . . . The HMD generates augmented reality (AR) content identifying the event and a direction of the location of the event relative to the location of the HMD and displays the AR content in the transparent display.” Ozery Abstract.
Ozery discloses audio “captured sensory dimension,” stating “The array of microphones enables the HMD to pick up ambient sound and identify a location of a particular sound based on the audio data from the array of microphones. The array of microphones is thus used to detect the event and to identify the event based on the audio data.” Ozery col. 4 lines 9-17.).
Regarding Claim 7, Ozery further teaches The method of claim 1, wherein performing the prioritization action comprises prioritizing rendering of the sensory feedback (
[BRI on the record] With respect to “prioritizing” something, the claim would be clearer, if Applicant clarify the reference, with respect to which a priority is given.
[Mapping Analyses]
“The higher the level of priority, the more likely the smart audio system will cause a display of the notification in the display of the HMD (within a field of view of the user) in addition to causing an auditory signal in the HMD (e.g., audio notification with an adjusted volume level based on background noise).” Ozery col. 2 lines 54-67.
For example, when AR content is displayed on HMD display, the rendering of AR content has been prioritized over the visual information the AR content obstructs or interfere with.).
Regarding Claim 8, Ozery further teaches The method of claim 7, wherein prioritizing rendering includes: enhancing the sensory feedback in at least one sensory dimension prior to the output for the sensory feedback (
“The HMD intercepts a notification and modifies the notification based on a priority level of the notification, the task-based context, and the ambient-based context. The modified notification comprises a combination of an audio notification and a visual notification. The visual notification includes an AR content. For example, the HMD may increase an audio output of speakers in the HMD to notify the user of the audio alert. The HMD may also supplement the audio output with displaying the visual notification in the transparent display of the HMD. The visual notification may include an escape route specific to the emergency related to the audio alert. For example, the HMD displays a first escape route based on a fire alarm and a different escape route based on a different type of alarm/event.” Ozery col. 5 lines 25-38.
The claimed “enhancing the sensory feedback” is mapped to “modif[ying] the notification based on a priority level of the notification.”
Alternatively or more specifically, including the “escape route” on display is an example for “enhancing the sensory feedback.” This mapping is consistent with Claim 9, Claim 8’s child.
“The higher the level of priority, the more likely the smart audio system will cause a display of the notification in the display of the HMD (within a field of view of the user) in addition to causing an auditory signal in the HMD (e.g., audio notification with an adjusted volume level based on background noise).” Ozery col. 2 lines 62-67.).
Regarding Claim 9, Ozery further teaches The method of claim 8, wherein enhancing the sensory feedback in at least one sensory dimension comprises rendering augmented sensory information or additional contextual information, or both, for output at the user device (
“The HMD may also supplement the audio output with displaying the visual notification in the transparent display of the HMD. The visual notification may include an escape route specific to the emergency related to the audio alert. For example, the HMD displays a first escape route based on a fire alarm and a different escape route based on a different type of alarm/event.” Ozery col. 5 lines 25-38.
the “escape route” is an example for “additional contextual information.”).
Regarding Claim 14, Ozery further teaches The method of claim 1, wherein the set of criteria includes one or more of: the event data, the location data, the one or more label, user preferences, and information about a user (
. “In another example embodiment, the smart audio system also defines a task hierarchy based on the priority of the task. For example, enterprise-specific tasks can be defined in the order in which the enterprise is interested in having the user/worker be interrupted. High priority tasks/actions are not to be interrupted using notifications. Low priority tasks allow for notifications interruption.” Ozery col. 3 lines 1-9.
The claimed “prioritization action” is mapped to prioritization of certain type of notification related to an event by a means that is more likely to catch a person’s attention, or prioritization of notification over a task being performed.
Ozery provides examples for relatively low and high priority events “based on a set of criteria,” stating “An example of a high priority task includes fixing a gas leak in a pipe. An example of a low priority task includes checking on a furnace filter.” Ozery col. 3 lines 7-9. Here, the priority is based on at least “event data.”
Here, if the user is a worker in an enterprise, “information about a uer,” it would have impact on the prioritization of the notification as well.).
Regarding Claim 18, Ozery further teaches The method of claim 1, wherein the event in the sensory environment is a potential environmental hazard, in the sensory environment, to a user of the user device (“If a danger is within an ‘unsafe distance’ to the user, the smart audio system engages a ‘distance to safety’ notification to notify the user of the location of the emergency and his proximity to it. The smart audio system can also play a notification that increases in intensity as the user moves closer to the location of the emergency, and decreases in intensity as the user moves away to a safe distance. When the user is out of harm's way, a unique audio notification is played to let the user know.” Ozery col. 3 lines 57-65.).
Claim 19 is substantially similar to Claim 1, and the rejection analyses for Claim 1 is applied to Claim 19. In addition, Claim 19 recites, “A system for processing event data for providing feedback, the system comprising: one or more processors; and a memory including instructions executable by said one or more processors to cause the system to: . . .” (Ozery figs. 1-2, 14.).
Claim 21 is substantially similar to Claim 1, and the rejection analyses for Claim 1 is applied to Claim 21. In addition, Claim 21 recites, “A non-transitory computer readable storage medium comprising instructions executable by one or more processors of a device, said instructions including instructions for to perform operations . . .” (Ozery figs. 1-2, 14).
Claims 4-6 and 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Ozery in view of Westberg and Bucsa as applied to Claim 1, in view of Saptharishi (US 20140085480 A1).
Regarding Claim 4, Ozery in view of Westberg and Bucsa teaches The method of claim 1, wherein the output for sensory feedback comprises:
prioritized processing of the event data (
Ozery teaches prioritized processing to notify of the event data based on the priority hierarchy of the tasks that the notification is going to interrupt, stating “the smart audio system also defines a task hierarchy based on the priority of the task. For example, enterprise-specific tasks can be defined in the order in which the enterprise is interested in having the user/worker be interrupted. High priority tasks/actions are not to be interrupted using notifications. Low priority tasks allow for notifications interruption. An example of a high priority task includes fixing a gas leak in a pipe. An example of a low priority task includes checking on a furnace filter.” Ozery col. 3 lines 1-9.
If the notification priority does not match the task priority of the task to be interrupted by the notification, the notification is not prioritized.),
prioritized rendering of the sensory feedback in at least one sensory dimension (“The higher the level of priority, the more likely the smart audio system will cause a display of the notification in the display of the HMD (within a field of view of the user) in addition to causing an auditory signal in the HMD (e.g., audio notification with an adjusted volume level based on background noise).” Ozery col. 2 lines 54-67. ).
Ozery in view of Westberg and Bucsa does not explicitly disclose prioritized transmission of communication related to the event data.
Saptharishi teaches prioritized transmission of communication related to the event data (Saptharishi teaches valuing packets of information related to an event based on an assigned priority for the event, stating “At the heart of content-aware infrastructure devices is a mechanism to analyze and value packets of information. The value of information is necessarily influenced by context provided by a human operator. But the raw sensor data has to be synthesized in a manner such that the assessment of value is possible. This synthesis is powered by a variety of analytics algorithms. Value is assessed based on user supplied rules conditional on the synthesized ‘summary’ of raw sensor data. The value assessment mechanism performs the following actions: (1) Synthesize a summary of sensor data with the appropriate amount of detail. (2) Based on user-supplied rules, detect the occurrence of events in the summary. Note that the rules are conditional on the descriptors in the summary. The descriptors are sometimes referred to as "metadata." (3) Based on the user-supplied rules, assign value to the detected events. (4) Based on the assigned value, the time when the event occurred and the probability with which the event occurs, assign a priority to the event. Note that priorities are used by the sensor network to guarantee QoS. The priorities, although related to the user-specified event values, are not always perfectly correlated.” Saptharishi ¶ 88.
Saptharishi further teaches transmission of the associated packets based on priority, stating “Based on the user-supplied rules, associated value and delivery time constraints, the packet (or set of packets) is assigned a priority. Packets are placed in the appropriate interface's FIFO buffer according to the assigned priorities. Some data might also be stored locally without further propagation of the data through the network. . . . (7) Transmit Packet(s): The processed packet is placed in regular send queue and the packet switch schedules delivery based on the specified bandwidth allocation and packet priority.” Saptharishi ¶ 117.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Saptharish’s prioritized transmission with Ozerv in view of Westberg and Bucsa. One of ordinary skill in the art would be motivated to prioritize important sensor information to be transmitted, so that the important sensor information would reach users faster. It would also reduce the amount of data to be transmitted, because some unimportant data may be dropped.
Regarding Claim 5, Ozery in view of Westberg and Bucsa teaches The method of claim 1.
Ozerv in view of Westberg and Bucsa does not explicitly teach wherein performing the one or more prioritization action includes prioritizing transmission of the event data.
Saptharishi teaches wherein performing the one or more prioritization action includes prioritizing transmission of the event data (
Saptharishi teaches valuing packets of information related to an event based on an assigned priority for the event, stating “At the heart of content-aware infrastructure devices is a mechanism to analyze and value packets of information. The value of information is necessarily influenced by context provided by a human operator. But the raw sensor data has to be synthesized in a manner such that the assessment of value is possible. This synthesis is powered by a variety of analytics algorithms. Value is assessed based on user supplied rules conditional on the synthesized ‘summary’ of raw sensor data. The value assessment mechanism performs the following actions: (1) Synthesize a summary of sensor data with the appropriate amount of detail. (2) Based on user-supplied rules, detect the occurrence of events in the summary. Note that the rules are conditional on the descriptors in the summary. The descriptors are sometimes referred to as "metadata." (3) Based on the user-supplied rules, assign value to the detected events. (4) Based on the assigned value, the time when the event occurred and the probability with which the event occurs, assign a priority to the event. Note that priorities are used by the sensor network to guarantee QoS. The priorities, although related to the user-specified event values, are not always perfectly correlated.” Saptharishi ¶ 88.
Saptharishi further teaches transmission of the associated packets based on priority, stating “Based on the user-supplied rules, associated value and delivery time constraints, the packet (or set of packets) is assigned a priority. Packets are placed in the appropriate interface's FIFO buffer according to the assigned priorities. Some data might also be stored locally without further propagation of the data through the network. . . . (7) Transmit Packet(s): The processed packet is placed in regular send queue and the packet switch schedules delivery based on the specified bandwidth allocation and packet priority.” Saptharishi ¶ 117.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Saptharish’s prioritized transmission with Ozerv in view of Westberg and Bucsa. One of ordinary skill in the art would be motivated to prioritize important sensor information to be transmitted, so that the important sensor information would reach users faster. It would also reduce the amount of data to be transmitted, because some unimportant data may be dropped.
Regarding 6, Ozery in view of Westberg, Bucsa, and Saptharishi teaches The method of claim 5, wherein prioritizing transmission of the event data comprises one or more of:
causing transmission of one or more communication packets associated with the event data prior to transmitting non-prioritized communication packets (
Saptharishi further teaches transmission of the associated packets based on priority, stating “Based on the user-supplied rules, associated value and delivery time constraints, the packet (or set of packets) is assigned a priority. Packets are placed in the appropriate interface's FIFO buffer according to the assigned priorities. Some data might also be stored locally without further propagation of the data through the network. . . . (7) Transmit Packet(s): The processed packet is placed in regular send queue and the packet switch schedules delivery based on the specified bandwidth allocation and packet priority.” Saptharishi ¶ 117.); and
causing transmission of one or more communication packets using a faster transmission resource.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Saptharish’s prioritized transmission with Ozerv in view of Westberg and Bucsa. One of ordinary skill in the art would be motivated to prioritize important sensor information to be transmitted, so that the important sensor information would reach users faster. It would also reduce the amount of data to be transmitted, because some unimportant data may be dropped.
Regarding Claim 11, Ozery in view of Westberg, Bucsa and Saptharishi teaches The method of claim 1, wherein performing the one or more prioritization action includes prioritizing processing of the event data, wherein prioritizing processing of the event data comprises one or more of:
causing processing of one or more communication packets associated with the event data prior to occur out of turn (
Saptharishi teaches processing of one or more communication packets to assess priority, stating “At the heart of content-aware infrastructure devices is a mechanism to analyze and value packets of information. The value of information is necessarily influenced by context provided by a human operator. But the raw sensor data has to be synthesized in a manner such that the assessment of value is possible. This synthesis is powered by a variety of analytics algorithms. Value is assessed based on user supplied rules conditional on the synthesized ‘summary’ of raw sensor data. The value assessment mechanism performs the following actions: (1) Synthesize a summary of sensor data with the appropriate amount of detail. (2) Based on user-supplied rules, detect the occurrence of events in the summary. Note that the rules are conditional on the descriptors in the summary. The descriptors are sometimes referred to as "metadata." (3) Based on the user-supplied rules, assign value to the detected events. (4) Based on the assigned value, the time when the event occurred and the probability with which the event occurs, assign a priority to the event. Note that priorities are used by the sensor network to guarantee QoS. The priorities, although related to the user-specified event values, are not always perfectly correlated.” Saptharishi ¶ 88.
Saptharishi further teaches transmission of the associated packets based on priority, causing packets to be “out of turn,” stating “Based on the user-supplied rules, associated value and delivery time constraints, the packet (or set of packets) is assigned a priority. Packets are placed in the appropriate interface's FIFO buffer according to the assigned priorities. Some data might also be stored locally without further propagation of the data through the network. . . . (7) Transmit Packet(s): The processed packet is placed in regular send queue and the packet switch schedules delivery based on the specified bandwidth allocation and packet priority.” Saptharishi ¶ 117.); and
causing processing of one or more communication packets to be added to a priority processing queue (Id.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Saptharish’s prioritized transmission with Ozerv in view of Westberg and Bucsa. One of ordinary skill in the art would be motivated to prioritize important sensor information to be transmitted, so that the important sensor information would reach users faster. It would also reduce the amount of data to be transmitted, because some unimportant data may be dropped.
Regarding Claim 12, Ozery in view of Westberg, Bucsa, and Saptharishi teaches The method of claim 1, wherein processing the event data further includes:
assigning a priority value to the event data, wherein the set of criteria includes the priority value (
Saptharishi teaches valuing packets of information related to an event based on an assigned priority for the event, stating “At the heart of content-aware infrastructure devices is a mechanism to analyze and value packets of information. . . . (3) Based on the user-supplied rules, assign value to the detected events. (4) Based on the assigned value, the time when the event occurred and the probability with which the event occurs, assign a priority to the event.” Saptharishi ¶ 88.
After Ozery is combined with Saptharishi, the set of criteria includes the priority value.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Saptharish’s prioritized transmission with Ozerv in view of Westberg and Bucsa. One of ordinary skill in the art would be motivated to prioritize important sensor information to be transmitted, so that the important sensor information would reach users faster and more reliably.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Ozery in view of Westberg, Bucsa, and Saptharishi as applied to Claim 12, in further view of Baek et al. (US 20080259956 A1)
Regarding Claim 13, Ozery in view of Westberg, Bucsa, and Saptharishi teaches The method of claim 12.
Ozery in view of Westberg, Bucsa, and Saptharishi does not explicitly disclose wherein the priority value is included in a transmission of event data to a remote processing resource or included in a received transmission of event data.
Baek teaches wherein the priority value is included in a transmission of event data to a remote processing resource or included in a received transmission of event data ( “A service priority SvcPriorty refers to a transmission priority included in the event message sending primitive EventMsgSend.” Baek ¶ 120. “A service priority (SP) field gives transmission priority to a transmission message and has 3 bits. Table 2 shows the priority of each transmission message.” Baek ¶ 59.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Baek’s transmission of priority information for events with Ozerv in view of Westberg, Bucsa, and Saptharishi. One of ordinary skill in the art would be motivated to prioritize the processing of important event data, so that the important event information would be processed faster and more reliably. Further, later data processing does not need to repeat the process that determines priority.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Ozeryin view of Westberg and Bucsa as applied to Claim 8, in view of Kasahara et al. (WO 2014162825 A1) and Spitzer et al. (US 20180136720 A1).
Regarding Claim 10, Ozery in view of Westberg and Bucsa teaches The method of claim 8, wherein the sensory feedback is a visual overlay output on a display of the user device (Ozery fig. 13), wherein the event in the sensory environment occurred outside of a current field-of-view of the display of the user device (
Ozery discloses, “The array of microphones enables the HMD to pick up ambient sound and identify a location of a particular sound based on the audio data from the array of microphones. The array of microphones is thus used to detect the event and to identify the event based on the audio data.” Ozery col. 4 lines 9-17.
The microphones detect an event based on sound, which could occur outside of a “current field-of-view” a user’s HMD.
Ozery provides examples for such events, stating “In one example embodiment, the HMD includes a transparent display. The HMD detects an event (e.g., an explosion, a siren, a gunshot, other types of sound) external to the HMD. In another example, the event includes a receiving an electronic communication that includes a warning notification (e.g., overheated engine).” Ozery col. 3 line 66 – col. 4 line 8.).
Ozery in view of Westberg and Bucsa does not explicitly disclose wherein enhancing the sensory feedback in at least one sensory dimension comprises increasing visual resolution of the sensory feedback relative to non-prioritized visual feedback that is outside the current field-of-view.
Kasahara teaches , which is inside the current field-of-view, in at least one sensory dimension and non-prioritized visual feedback that is outside the current field-of-view (
PNG
media_image5.png
482
684
media_image5.png
Greyscale
PNG
media_image6.png
468
560
media_image6.png
Greyscale
Kasahara provides examples for the “sensory feedback”: 1230a, 1230b, and 1220, that could indicate the presence of an event, pan being used in cooking, that occur outside and inside of the field of view, stating “In the image 1200a and the image 1200b, since the pan (PAN) is out of the visible range of the user, the direction display 1230a and the direction display 1230b indicating the direction in which the pan is present are displayed. When the user moves the display range of the image 1200 according to the direction display 1230, captures the pan in the display range in the image 1200c, and puts the pan into the pointer 1210, the comment 1220 is displayed for the first time. An image 1200c at this time is separately shown in FIG.” Kasahara p. 22.
After, Ozery is combined with Kasahara, Kasahara’s “sensory feedback” would be used for events disclosed by Ozery. Also note Ozery’s fig. 13
PNG
media_image3.png
402
464
media_image3.png
Greyscale
shows an arrow that is similar to Kasahara fig. 31 1230a and 1230b, and both arrows point to a location.
AR elements like 1230a, 1230b are for events outside of field of view.
AR element like 1220 is for event inside of the field of view, and 1220 is closer to the focus area of the field of view.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Kasahara’s with Ozerv in view of Westberg and Bucsa. One of ordinary skill in the art would be motivated to show a HMD user where the location of an event may be, even if the event has occurred outside of the field of view.
Ozery in view of Westberg, Bucsa, and Kasahara does not explicitly disclose wherein enhancingthe sensory feedback, which is inside the current field-of-view comprises increasing visual resolution of the sensory feedback relative to non-prioritized visual feedback that is outside the current field-of-view.
Spitzer teaches wherein enhancingthe sensory feedback, which is inside the current field-of-view comprises increasing visual resolution of the sensory feedback relative to non-prioritized visual feedback that is outside the current field-of-view (
“A foveated display system includes a rendering device including at least one graphics processing unit (GPU) to render a foveal region and a peripheral region of a first image, wherein the foveal region has a higher resolution than the peripheral region.” Spitzer Abstract.
This mapping is consistent with the specification ¶ 121: “For example, visual information for an event that occurs outside of a user's current field of view can be enhanced so that when the user turns their head to look at the event in the environment, a higher quality or resolution image or overlay (than otherwise would have, for example, due to foveated rendering) is ready to be displayed instantly and without delay to due to latency.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Spitzer’s foveated display with Ozerv in view of Westberg, Bucsa, and Kasahara. One of ordinary skill in the art would be motivated to maintain the quality of a displayed image, while reducing the computation cost. A user cannot see the peripheral areas clearly in a HMD display, and therefore, there is no need to waste computation resources to render for the peripheral areas in higher resolutions.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Ozery in view of Westberg and Bucsa as applied to Claim 1, in further view of Fedorov et al. (US 20200351380 A1).
Regarding Claim 15, Ozery in view of Westberg and Bucsa teaches The method of claim 1.
However, Ozery in view of Westberg and Bucsa does not explicitly disclose wherein the edgecloud performs local priority processing of the event data based at least on one or more of local processing resources, remote processing resources, and a transmission latency between local processing resources and remote processing resources.
Fedorov teaches
wherein determining whether to perform a prioritization action includes
determining whether to perform local priority processing of the event data based at least on one or more of local processing resources, remote processing resources, and a transmission latency between local and remote resources (
Fedorov teaches “determining whether to perform local . . . processing,” stating “In one example, the decision of where to process may be ‘comparison-based.’ The methods may include determining the cost of performing actions at the edge and in the cloud and selecting the location with lowest cost for performing the actions. . . . Even when defined in the second sense, the term ‘cost’ (when referring to latency, resource availability, performance requirement, processing precision, status of edge resources, status of cloud resources, bandwidth, computing power, and other processing factors) may be evaluated by the systems and methods of the present disclosure in monetary terms in order to determine a total cost (e.g., in U.S. dollars) of local processing versus remote processing. The “cost” of these various processing functions may be set or may be variable, according to different embodiments. For example, if latency issues are very important in one type of IoT application (e.g., where quick response is needed to get relevant information to the IoT devices or users, such as a vehicle response type of action), the ‘cost’ of remote processing may be given a very high rate, thereby essentially eliminating the viability of remote processing in this case.” Fedorov ¶ 102.
After Ozery in view of Westberg and Bucsa is combined with Fedorov, “prioritization action” is performed locally or remotely, because Ozery already teaches “prioritization action.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Fedorov’s strategy to allocate computation tasks with Ozerv in view of Westberg and Bucsa. One of ordinary skill in the art would be motivated to lower the cost of computation and potentially increase the speed.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
PAUL et al. (US 20200110935 A1): AUGMENTED REALITY SYSTEM TO MAP AND VISUALIZE SENSOR DATA, which is similar to Applicant’s claimed invention as shown in fig. 6 about visual alert for a gas leak
PNG
media_image7.png
410
522
media_image7.png
Greyscale
Thyagaturu et al. (US 20210149707 A1) similar to feature in Claim 15.
Thyagaturu recites, “ In some examples, the IPUs 102 of FIG. 1 can provide orchestration to decide where to execute a workload based on available resources (e.g., services and devices) and by considering service level agreements and latencies, to determine whether resources (e.g., CPU, GPU, XPU, storage, memory) are to be allocated from the local host or from a remote host or pooled resource.” Thyagaturu ¶ 22.
Bachhuber, Christoph, et al. "Edge cloud-based augmented reality." 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2019. (Year: 2019)
As the title shows that the article is related to claimed feature related to edgecloud for augmented reality.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZHENGXI LIU whose telephone number is (571)270-7509. The examiner can normally be reached M-F 9 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at 571-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZHENGXI LIU/Primary Examiner, Art Unit 2611
Prosecution Insights