Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
Claim 4 recites, in part, a limitation for “a signal from the global neural network to the local neural network being tailored to the specific group of wearers and, optionally, not comprising any information related to a group identity of the specific group of wearers.” The presence of “optionally”, as underlined in the aforementioned limitation, has the effect of clarifying that the language that follows is not required. Hence, for purposes of present examination, the Examiner will not address this language. If Applicants intend for this language to be considered, then the scope of the language should be changed so that it is not optional as it presently reads.
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: “input module” in claims 1 and 5 (and claims depending therefrom), and “sensing module” in claim 7.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. See, e.g., specification [0044]-[0047] and [0073] discussing processor element, and [0089] discussing features of the local neural network situated in relation to the input module such that they are implemented using “a dedicated processing circuit”, and further, [0086] discussing physical sensors described in the implementation of the sensing module.
If Applicants do not intend to have these limitations interpreted under 35 U.S.C. 112(f), Applicants may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 7 is rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
In particular, the claim recites, in part, “wherein the input module comprises a sensing module adapted to sense: environment data, for instance in a list comprising one or more of the following: luminosity data, distance data and frame boxing data, and/or
wearer data, for instance in a list comprising one or more of the following: activity data, sensitivity data, laterality data, and posture data.”
The underlined language “for instance” is being read as approximating “for example” as might be used colloquially/informally in conversation, which is to say that “environment data” and/or “wearer data” are required by the claim but not necessarily the specific instances of it that follow in the respective lists. Accordingly, the use of such language makes it unclear whether the listed items are required as constituting “one or more” of the data categories as recited, or whether the listed items are more ornamental in the sense that they provide examples that might comply but are not strictly required so long as some environmental data and/or wearer data is present.
The Examiner recommends a clarifying amendment to remove the underlined language “for instance”, if Applicants intend for the listed items for one or more of either/or environmental data and/or wearer data to be required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 4-12 are rejected under 35 U.S.C. 103 as being unpatentable over CA 3133317 A1 (“Abou Shousha”) in view of U.S. Patent Application Publication No. 2023/0128422 (“Li”).
Regarding claim 1, ABOU SHOUSHA teaches A communication system for a head-worn device having an active function (FIG. 1A’s system for facilitating modification relating to a user’s vision, per [068], equivalent to the recited “communication system”, where the user is associated with a client device that may be a wearable device per [068] and more specifically may be a spectacles device ([078]-[082], [085]-[095], and [0100]-[0103]), i.e., equivalent to the recited “head-worn device”, and where the device/spectacle is understood to monitor and improve the user’s visual experience ([003], [005], [072], [074]-[077], [080] (“The spectacles device 170 or associated vision system may be further configured to then correct and/or enhance the images, which may be in a customized manner based on the optical pathologies of the subject.”), and [088], where the Examiner equates the view correction/augmentation as performed in various manners as described to be “an active function” as recited)), comprising:
an interface capable of transmitting and receiving signals between a local neural network and a server/cloud’s model (FIG. 1A’s system, or any more specific instance of it relating to a spectacle, is situated such that the device/spectacle is in communication with a server ([068], [088]), and where the device/spectacle is understood to use a model that could be implemented via a neural network ([072]-[074]) and where the model may be stored and synchronized both locally and remotely/centrally in the cloud ([097])),
wherein:
the local neural network enables control of the active function using information gathered by an input module and related to a wearer of the head-worn device or his surrounding environment ([080] teaching “The spectacles device 170 or associated vision system may be further configured to then correct and/or enhance the images, which may be in a customized manner based on the optical pathologies of the subject.” (also [083]), i.e., a per-user vision correction/augmentation, which is achieved in part by modelling associated with the user and the prediction results therefrom ([097]: “In some embodiments, one or more prediction models may be trained or configured for a user or a type of device (e.g., a device of a particular brand, a device of a particular brand and model, a device having a certain set of features, etc.) and may be stored in association with the user or the device type. … Based on the eye characteristics or environmental characteristics currently detected, the corresponding set of modification parameters or functions may be obtained and used to generate the enhanced presentation of the live image data”)),
While the aforementioned reference clearly teaches a framework of local models personalized for particular users, where the local models and users are situated at edges of a network relative to a centralized server, it does not explicitly teach federated learning with an explicitly-defined global neural network. To that end, Abou Shousha does not teach that the interface, as addressed above, is capable of transmitting and receiving signals between a local neural network and specifically a global neural network, and also that the transmitted and received signals facilitate participation of the local neural network in a federated learning process with the global neural network. Rather, the Examiner relies upon LI to teach what Abou Shousha otherwise lacks, see e.g., Li’s augmented/virtual reality system ([0002]), where users wear headsets or smart glasses ([0054], [0094]), and for which the modelling extends to computer vision and specifically object detection by using eye tracking/monitoring ([0078]), and for which the model contemplates local and global training aspects ([0102]) amenable to the framework’s explicit consideration of using federated learning to train, develop, and process in relation to its models on both local and global/central levels ([0087]).
Both Abou Shousha and Li relate to vision-based head-worn user device experiences that leverage local and remote/centralized model considerations to provide benefits/advantages relating to a user’s visual experience/capabilities. Hence, the references are similarly directed and therefore analogous. The Examiner notes that Abou Shousha contemplates model training in a capacity that scales from the particular user to many users, see e.g., [0259]. Li more formally contemplates federated learning, which provides a mechanism so that a particular model that is local can benefit from a model more widely/broadly trained. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to more formally extend Abou Shousha’s framework, using some of Li’s explicit federated learning teachings, so that Abou Shousha as modified can benefit its local users with some transferred knowledge from a model that is more widely and broadly trained as might be appropriate.
Regarding claim 2, Abou Shousha in view of Li teach the communication system of claim 1, as discussed above. The aforementioned references teach the additional limitations wherein a signal from the local neural network encapsulates a set of N local weights, N>1, representing a relationship between training data and their respective labels, wherein the training data are information gathered by the input module and the labels correspond to various states of the active function (Abou Shousha’s [0073] discussing the labeling of inputs to train the model, by which the updating of model configuration, including its weights, based on assessment of outputs/predictions, are adjusted “to reconcile differences between the neural network's prediction and the reference feedback”, with [0102] clarifying that such a model may be local / locally trained, and with [0072] clarifying that the modelling may be prediction modelling to determine vision defects and modification profiles to correct therefor, e.g., as discussed in [097] as eye characteristics and environmental characteristics for such a modification profile (i.e., equivalent to states of an active function for a spectacle as taught)). The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 4, Abou Shousha in view of Li teach the communication system of claim 1, as discussed above. The aforementioned references teach the additional limitations wherein the federated learning process is a personalized federated learning process (Li’s augmented/virtual reality system features modeling relating to computer vision and specifically object detection by using eye tracking/monitoring ([0078]), and for which the model contemplates local and global training aspects ([0102]) amenable to the framework’s explicit consideration of using federated learning to train, develop, and process in relation to its models on both local and global/central levels ([0087])) defined by:
the global neural network having access to wearer data indicating that the wearer of the head-worn device belongs to a specific group of wearers (Abou Shousha’s [0097] discussing the association of models and related profiles, as stored in the cloud, with differing groups of users, i.e., group of users wearing the device/spectacle as taught), and
a signal from the global neural network to the local neural network being tailored to the specific group of wearers (Li’s [0087] discussing a federated learning approach that furnishes parameters, i.e., weights, to client devices and hence their local models, which the Examiner believes is equivalent to the grouping of users as discussed above per Abou Shousha’s [0097]) and, optionally, not comprising any information related to a group identity of the specific group of wearers.
The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 5, the claim includes the same or similar limitations as discussed above in relation to claim 1, and is therefore rejected under the same rationale.
Regarding claim 6, Abou Shousha in view of Li teach the head-worn device of claim 5, as discussed above. The aforementioned references teach the additional limitations wherein the input module comprises a human-machine interface adapted to receive wearer inputs from the wearer of the head-worn device (Abou Shousha’s [097]: “eye characteristics” for a particular user / wearer of device/spectacle). The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 7, Abou Shousha in view of Li teach the head-worn device of claim 5, as discussed above. The aforementioned references teach the additional limitations wherein the input module comprises a sensing module adapted to sense:
environment data, for instance in a list comprising one or more of the following: luminosity data, distance data and frame boxing data (Abou Shousha’s [097]: “environmental characteristics”), and/or
wearer data, for instance in a list comprising one or more of the following: activity data, sensitivity data, laterality data, and posture data (Abou Shousha’s [097]: “eye characteristics”, as discussed just above per claim 6).
The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 8, Abou Shousha in view of Li teach the head-worn device of claim 5, as discussed above. The aforementioned references teach the additional limitations wherein the head-worn device comprises an optical lens and the active function is an optical function of the optical lens (Abou Shousha’s [072]: “In some embodiments, one or more prediction models may be used to facilitate determination of vision defects (e.g., light sensitivities, distortions, or other aberrations), determination of modification profiles (e.g., correction/enhancement profiles that include modification parameters or functions) to be used to correct or enhance a user's vision, generation of enhanced images (e.g., derived from live image data), or other operations.” (with [074]-[075] discussing in more detail the types of optical adjustments made, including transparency/opaqueness changes, and relating thereto: positioning, shapes, sizes, etc.), and where the wearable device may explicitly be/include a spectacle as discussed above per claim 1 (and for example, [078]), thereby at least inherently having a lens element). The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 9, Abou Shousha in view of Li teach the head-worn device of claim 8, as discussed above. The aforementioned references teach the additional limitations wherein the optical function is a transmission function (Abou Shousha’s [074]-[075] discussing opaqueness/transparency, which reads on a transmission of light/information as captured, processed, and then rendered/presented), an optical power function or a defocus function. The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 10, Abou Shousha in view of Li teach the head-worn device of claim 8, as discussed above. The aforementioned references teach the additional limitations wherein the head-worn device comprises a screen and the active function is a function of the screen (Abou Shousha’s [078] teaching that the spectacle may have a display screen, such as to facilitate the presentation of augmented image/video to the user/wearer of the spectacle). The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 11, the claim includes the same or similar limitations as discussed above in relation to claim 1, and is therefore rejected under the same rationale.
Regarding claim 12, the method of claim 11 is clarified to be carried out in relation to A non-transitory computer-readable storage medium having stored thereon a computer program comprising instructions … executed by a processor, which is taught by Abou Shousha’s [095] discussing program memory implementations that are executable by a microprocessor that extend into the basic processing steps broadly contemplated by the framework.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Abou Shousha in view of Li and further in view of U.S. Patent Application Publication No. 2021/0065649 (“Saa-Garriga”).
Regarding claim 3, Abou Shousha in view of Li teach the communication system of claim 2, as discussed above. While the aforementioned references do teach a neural network type model with weights and even a federated learning framework where models and weights are transferrable from a centralized agent to a local/edge agent (see the rejection to claim 1, for example), they do not teach specifically wherein a signal from the global neural network encapsulates M global weights, N>M≥1, and each received global weight is utilized to replace a corresponding weight of the set of local weights, particularly the limitation’s aspect that limits the number of global weights relative to the number of local weights. Rather, the Examiner relies upon SAA-GARRIGA to teach what Abou Shousha etc. otherwise lack, see e.g., Saa-Garriga’s [0087] discussing the selection of only a subset of weights for transfer between global and local model instances in a federated learning framework that is comparable to Li’s and Abou Shousha as modified.
Like Abou Shousha, Saa-Garriga relates to display-based modelling. Hence, the references are similarly directed and therefore analogous. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Abou Shousha in view of Li specifically, for Li’s more formal federated learning approach, to transfer only some of the weights as selected between server and edge elements of the framework, as Saa-Garriga teaches, with a reasonable expectation of success, to streamline the requisite data transfer in view of bandwidth, network, and privacy considerations known in the state of the art.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicants’ disclosure:
US 20180345128 A1 Ahmed
US 20230266589 A1 Eaton
US 20220114616 A1 Ingram
US 20220138626 A1 BI
CN 110023814 B Amayeh
WO 2021094777 A1 Cashmore
CN 115510974 A Chen
CN 116233954 A Teng
Non-Patent Literature “Smart Glasses-Based Personnel Proximity Warning System for Improving Pedestrian Safety in Construction and Mining Sites” (Baek)
Non-Patent Literature “Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge Based Framework” (Qu)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHOURJO DASGUPTA whose telephone number is (571)272-7207. The examiner can normally be reached M-F 8am-5pm CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached at 571 272 4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHOURJO DASGUPTA/Primary Examiner, Art Unit 2144
Prosecution Insights