DETAILED ACTION
Claims 1-2, 7-8, 11-16, 19-25, 27, 34-35, 40-41, 44, and 67 are the present claims hereby under consideration.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 19 and 27 are objected to because of the following informalities:
Claim 19 lines 2 it appears that “the plurality of estimated body reference keypoints” should read “the plurality of body reference keypoints”
Claim 27 it appears that “detecting one or more characteristics” should read “detecting the one or more characteristics”
Appropriate correction is required.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 14, 19, 22, 24, 34-35, 40-41, 44, and 67 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 14 recites “a plurality of body reference keypoints” but it is unclear if this limitation is the same as, related to, different from, or a subset of “one or more characteristics of the patient’s body” of claim 1. For the purposes of this examination, these limitations are interpreted as referring to the same bodily characteristics.
Claim 19 recites “obtaining the plurality of estimated body reference keypoints comprises processing the one or more images using a machine-learning model for estimating pose” which appears to indicate that the machine learning model is configured to estimate a pose of the subject. It is unclear if the pose of the subject is considered the same as the “body reference keypoints”. It is further unclear how a machine learning model configured to perform one operation (the estimation of pose) is being utilized to carry out the estimation of the body reference keypoints. For the purposes of this examination, the limitation is interpreted as a machine learning model being used to determine the keypoints.
Claims 22 and 24 each recite the limitation "the one or more user interface objects" in line 3. There is insufficient antecedent basis for this limitation in the claim.
Claim 34 recites “one or more indicators” in line 5 but it is unclear what the metes and bounds of an indicator entail and/or if the limitation is limited to physical components. In particular, it is unclear if non-physical elements may be considered “indicators” and is so it is unclear how such elements could be considered part of the computing device. For the purposes of this examination, the indicators are interpreted as the physical elements which may provide indications to the user rather than the indicators themselves. This rejection is similarly applied to claim 67 which further recites indicators as a physical component of a device.
Claims 35, 40-41, and 44 are rejected by virtue of their dependency on claim 34.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-2, 7-8, 11-16, 19-25, 27, 34-35, 40-41, 44, and 67 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Claims 1-2, 7-8, 11-16, 19-25, 27, 34-35, 40-41, 44, and 67 are directed to a method of processing images to generate guidance using a computational algorithm, which is an abstract idea. Claims 1-2, 7-8, 11-16, 19-25, 27, 34-35, 40-41, 44, and 67 do not include additional elements that integrate the exception into a practical application or that are sufficient to amount to significantly more than the judicial exception for the reasons provided below which are in line with the 2014 Interim Guidance on Patent Subject Matter Eligibility (Federal Register, Vol. 79, No. 241, p 74618, December 16, 2014), the July 2015 Update on Subject Matter Eligibility (Federal Register, Vol. 80, No. 146, p. 45429, July 30, 2015), the May 2016 Subject Matter Eligibility Update (Federal Register, Vol. 81, No. 88, p. 27381, May 6, 2016), and the 2019 Revised Patent Subject Matter Eligibility Guidance (Federal Register, Vol. 84, No. 4, page 50, January 7, 2019) and the 2024 Update on Subject Matter Eligibility (Federal Register, Vol 89, No. 137, page 58128, July 17, 2024).
The analysis of claim 1 is as follows:
Step 1: Claim 1 is drawn to a process
Step 2A – Prong One: Claim 1 recites an abstract idea. In particular, claim 1 recites the following limitations:
[A1] detecting one or more characteristics of the patient's body in the one or more images
[B1] determining a target location for placing the medical device on the patient's body based on the detected one or more characteristics
[C1] detecting a location of the medical device
[D1] generating guidance for moving the medical device from the detected location of the medical device to the determined target location
[E1] providing the guidance to the patient via one or more indicators of the computing device
These elements [A1]-[E1] of claim 1 are drawn to an abstract idea since they involve a mental process that can be practically performed in the human mind including observation, evaluation, judgment, and opinion and using pen and paper; and/or they involve methods of organizing human activity such as managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions). In particular, the method is drawn towards guidance that would be provided by a clinician in a clinician setting. The clinician would view the patient and note characteristics of their body, determine a placement of a medical device based on the characteristics of the body, determine where the medical device is currently located, determine how to navigate the device to the proper location, and guide the patient in moving the device to the desired location
Step 2A – Prong Two: Claim 1 recites the following limitations that are beyond the judicial exception:
[A2] a medical device
[B2] a computing device
[C2] acquiring one or more images of the patient
[D2] one or more cameras of the computing device
These elements [A2]-[D2] of claim 1 do not integrate the exception into a practical application of the exception. In particular, the elements [C2] and [D2] are merely adding insignificant extra-solution activity to the judicial exception, i.e., mere data gathering at a higher level of generality - see MPEP 2106.04(d) and MPEP 2106.05(g). Furthermore, the element [B2] is merely an instruction to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.04(d) and MPEP 2106.05(f). Further, the element [A2] merely generally links the use of the judicial exception to a particular technological environment or field of use – see MPEP 2106.05(h).
Step 2B: Claim 1 does not recite additional elements that amount to significantly more than the judicial exception itself. In particular, the recitation “acquiring one or more images of the patient via one or more cameras of the computing device” is merely insignificant extrasolution activity to the judicial exception, e.g., mere data gathering in conjunction with the abstract idea that uses conventional, routine, and well known elements or simply displaying the results of the algorithm that uses conventional, routine, and well known elements. In particular, the data acquirer is nothing more than a camera in a computing device which may be a smartphone, laptop, or other smart device such as a smart mirror. Such devices are well known in the art as evidenced by Applicant’s lack of a particular description in regards to the structure and/or operation of the device and further evidenced by:
U.S. Patent Application Publication No. US 20200348896 A1 (Liu) discloses that smart mirrors are conventional (paragraph 0003 of Liu);
U.S. Patent Application Publication No. US 20210205688 A1 (Liao) discloses that smart mirrors are conventional (paragraph 0004 of Liao);
U.S. Patent Application Publication No. US 20240065554 A1 (Serval) discloses that existing smart mirrors typically include a camera and display integrated with a mirror (paragraph 0003 of Serval); and
U.S. Patent Application Publication No. US 20220319707 A1 (Rhode) lists a smart mirror, smart phone, laptop, and other devices as typical, camera enabled smart devices (paragraph 0009 of Rhode).
U.S. Patent Application Publication No. US 20220262010 A1 (Shpuza) lists a smart mirror and other smart devices as typical devices which include conventional computing components and cameras (paragraph 0059 of Shpuza).
Further, the element [B2] does not qualify as significantly more because this limitation is simply appending well-understood, routine and conventional activities previously known in the industry, specified at a high level of generality, to the judicial exception, e.g., a claim to an abstract idea requiring no more than a generic computer to perform generic computer functions that are well-understood, routine and conventional activities previously known in the industry (see Electric Power Group, 830 F.3d 1350 (Fed. Cir. 2016); Alice Corp. v. CLS Bank Int’l, 110 USPQ2d 1976 (2014)) and/or a claim to an abstract idea requiring no more than being stored on a computer readable medium which is a well-understood, routine and conventional activity previously known in the industry (see Electric Power Group, 830 F.3d 1350 (Fed. Cir. 2016); Alice Corp. v. CLS Bank Int’l, 110 USPQ2d 1976 (2014); SAP Am. v. InvestPic, 890 F.3d 1016 (Fed. Circ. 2018)).
Additionally, the element [A2] merely served to link the method to uses in the medical field. The “medical device” is not meaningfully described and may include any well-known medical device.
In view of the above, the additional elements individually do not integrate the exception into a practical application and do not amount to significantly more than the above-judicial exception (the abstract idea). Looking at the limitations as an ordered combination (that is, as a whole) adds nothing that is not already present when looking at the elements taking individually. There is no indication that the combination of elements improves the functioning of a computer, for example, or improves any other technology. There is no indication that the combination of elements permits automation of specific tasks that previously could not be automated. There is no indication that the combination of elements includes a particular solution to a computer-based problem or a particular way to achieve a desired computer-based outcome. Rather, the collective functions of the claimed invention merely provide conventional computer implementation, i.e., the computer is simply a tool to perform the process.
Claims 2, 7-8, 11-16, 19-25, and 27 depend from claim 1, and recite the same abstract idea as claim 1. Furthermore, these claims only contain recitations that further limit the abstract idea (that is, the claims only recite limitations that further limit the algorithm), with the following exceptions:
Claim 2: a smart mirror;
Claim 11: generating a photogrammetric model; and
Claims 19-20 and 27: using a machine learning model.
Claims 24-25: the generation of “user interface objects”
Each of these claims limitations does not integrate the exception into a practical application. In particular, the elements of claim 2 are merely limiting the method to be executed on a smart mirror but such smart mirrors are well-known, routine, and conventional as evidenced by Liu, Liao, Serval, Rhode, and Shpuza as presented above.
Additionally, the limitations of claim 11 drawn towards the generation of a photogrammetric model and of claims 19-20 and 27 drawn towards the use of machine learning models are not considered to amount to significantly more than the abstract idea because these limitations are drawn towards the computer implementation/automation of an abstract mental process of observing and evaluating a patient based on the observations which is what a physician typically does with a patient in a diagnostic setting. In particular, the photogrammetric model is merely a computer implementation to represent what a physician sees when observing a patient, The subsequent calculations made using the model are merely the computer implementation of a clinician making decisions in regards to where a device can be placed with respect to the observed patient’s morphology. In particular, the evaluation of normal surface vectors and gradient surface vectors are limitations drawn towards the computer implementation of a clinician evaluating the curvature of the patient’s body and determining which areas on the body have curves, folds, or other skin features not suitable for device placement.
Additionally, the limitations of claims 19-20 and 27 drawn towards the use of machine learning models to determine certain features are merely recitations drawn towards the computer implementation/automation of a clinician’s ability to identify keypoints and skin characteristics on the body based on their observations.
Finally, the limitations of claims 24-25 are considered to be drawn towards the computer implementation/automation of the clinician providing feedback to the user on how to move the medical device such as by pointing their fingers in particular directions. The “user interface objects” are not considered to amount to anything more than the generic display of the algorithm’s results.
It is further noted that the identification of device reference keypoints as recited in claim 23 is not considered to add an additional element to the device because the identified points may be any points on a device and are thus inherently present in any physical object.
In view of the above, the additional elements individually do not integrate the exception into a practical application and do not amount to significantly more than the above-judicial exception (the abstract idea). Looking at the limitations of each claim as an ordered combination in conjunction with the claims from which they depend (that is, as a whole) adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer, for example, or improves any other technology. There is no indication that the combination of elements permits automation of specific tasks that previously could not be automated. There is no indication that the combination of elements includes a particular solution to a computer-based problem or a particular way to achieve a desired computer-based outcome. Rather, the collective functions of the claimed invention merely provide conventional computer implementation, i.e., the computer is simply a tool to perform the process.
Claims 34-35, 40-41, and 44 recite the same abstract idea as claim 1 and contain only additional limitations that are outside the abstract idea that have already been addressed in the above rejection of claims 1-2, 7-8, 11-16, 19-25, and 27 and are thus rejected on the same grounds presented above with the following exceptions:
Claim 34 recites the additional limitations of a computing device comprising one or more cameras, indicators, and processors.
These additional elements do not qualify as significantly more because this limitation is simply appending well-understood, routine and conventional activities previously known in the industry, specified at a high level of generality, to the judicial exception, e.g., a claim to an abstract idea requiring no more than a generic computer to perform generic computer functions that are well-understood, routine and conventional activities previously known in the industry (see Electric Power Group, 830 F.3d 1350 (Fed. Cir. 2016); Alice Corp. v. CLS Bank Int’l, 110 USPQ2d 1976 (2014)) and/or a claim to an abstract idea requiring no more than being stored on a computer readable medium which is a well-understood, routine and conventional activity previously known in the industry (see Electric Power Group, 830 F.3d 1350 (Fed. Cir. 2016); Alice Corp. v. CLS Bank Int’l, 110 USPQ2d 1976 (2014); SAP Am. v. InvestPic, 890 F.3d 1016 (Fed. Circ. 2018)). Thus, these elements require nothing more than a typical computer performing typical computer functions.
Claim 35 limits these elements to being implemented with the addition of a mirror. As evidenced in the above rejection of claims 1-2, smart mirrors are routine, conventional, and/or well-known and may include all the conventional elements of a typical computer. Thus, these additional limitations do not amount to significantly more than the abstract idea itself as they are drawn towards a computer implementation of the abstract idea.
The analysis of claim 67 is as follows:
Step 1: Claim 67 is drawn to a machine
Step 2A – Prong One: Claim 67 recites an abstract idea. In particular, claim 67 recites the following limitations:
[A1] receive one or more instructions for providing guidance to a patient for placing the medical device on the patient’s body
[B1] providing the guidance to the patient via one or more indicators of the medical device
These elements [A1]-[B1] of claim 67 are drawn to an abstract idea since they involve a mental process that can be practically performed in the human mind including observation, evaluation, judgment, and opinion and using pen and paper; and/or they involve methods of organizing human activity such as managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions). In particular, the method is drawn towards guidance that would be provided by a clinician in a clinician setting.
Step 2A – Prong Two: Claim 67 recites the following limitations that are beyond the judicial exception:
[A2] a medical device comprising one or more indicators and one or more processors and in communication with an external processor
This element [A2] of claim 67 does not integrate the exception into a practical application of the exception. In particular, the element [A2] is merely an instruction to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.04(d) and MPEP 2106.05(f). In particular, the “medical device” is not limited to having any form of sensor or treatment mechanism. The medical device is described only as including indicators and processors and as being placed on the patient’s body. The “medical device” may thus include a conventional smartphone which includes a screen, processor, and wireless communication capabilities. The smartphone and can be placed on the body.
Step 2B: Claim 67 does not recite additional elements that amount to significantly more than the judicial exception itself. In particular, the medical device is not meaningfully described and may be a conventional smartphone configured to receive and display the provided instructions.
In view of the above, the additional elements individually do not integrate the exception into a practical application and do not amount to significantly more than the above-judicial exception (the abstract idea). Looking at the limitations as an ordered combination (that is, as a whole) adds nothing that is not already present when looking at the elements taking individually. There is no indication that the combination of elements improves the functioning of a computer, for example, or improves any other technology. There is no indication that the combination of elements permits automation of specific tasks that previously could not be automated. There is no indication that the combination of elements includes a particular solution to a computer-based problem or a particular way to achieve a desired computer-based outcome. Rather, the collective functions of the claimed invention merely provide conventional computer implementation, i.e., the computer is simply a tool to perform the process.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 7-8, 11, 14-16, 23-25, 34, 40-41, and 44 are rejected under 35 U.S.C. 103 as being unpatentable over Talgorn US Patent Application Publication Number US 20200121249 A1 hereinafter Talgorn in view of Fiedler US Patent Application Publication Number US 20190076604 A1 hereinafter Fiedler.
Regarding claim 1, Talgorn discloses a method for guiding a patient to place a medical device on the patient's body using a computing device (Abstract: the method and apparatus comprising a processor), the method comprising:
acquiring one or more images of the patient via one or more cameras of the computing device (Paragraphs 0052 and 0062: at least one acquired image from one or more cameras);
detecting one or more characteristics of the patient's body in the one or more images (Paragraphs 0053 and 0062: the images are analyzed to recognize body parts; Paragraph 0066-0067: the method may further involve identifying markers such as skin fold, pores, birthmarks, and other body markers including markers that are acquired during removal of a previously placed device)
determining a target region for placing the medical device on the patient's body based on the detected one or more characteristics (Paragraphs 0053 and 0062: The recognized body parts are used to identify the body part at which to place the device);
detecting a location of the medical device (Paragraph 0060: detecting the location of the wearable device);
generating guidance for moving the medical device from the detected location of the medical device to the determined target region (Paragraphs 0057-0060: the guidance to position the wearable device in relation to the identified body part is provided and must therefore also be generated); and
providing the guidance to the patient via one or more indicators of the computing device (Paragraphs 0057-0060: the guidance is provided to the user through the device itself and/or through an external device such as a smart mirror).
Talgorn fails to further disclose the method including determining a particular target location and guiding the device to the target location.
Fiedler teaches systems and methods encourage users to rotate injection sites and avoid lipodystrophy. Sleeves and/or lost-tooth gear dials and/or microswitches in or on injection pens or their caps, on vials, and on other portable devices manually adjust an indicator before or after an injection to show a current or next injection site in accordance with a site rotation plan. Injected medicine packaging and related printed indicia encourage site rotation. Optical devices employing optical mouse or projection technology help locate and/or distribute injection sites within a body area. A mobile phone app tracks injections and locations to select next injection site, and can use imaging to locate a target injection site and optionally diagnose lipodystrophic conditions and record them (Abstract). Thus, Fiedler falls within the same field of endeavor as Applicant’s invention.
Fiedler teaches the use of optical sensors to capture images of the body and identify references points and injection sites. The system logs the injection sites including their time and may indicate that an injection site is not usable until a certain amount of time has passed or other criteria are met (Paragraphs 0184-0191). The system may determine where valid injection sites are and recommend a target site location based on a number of configurable criteria including proximity from last injection site, amount of time that has passed since the previous injection at that site, degree of lipodystrophy, and other factors (Paragraphs 0192-0194).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to implement the selection of a particular target location as taught by Fiedler because Fielder teaches that injection sites may need to be rotated to prevent adverse effects and further contemplates that other criteria may need to be met before a particular site may be used again (Fiedler: Paragraph 0191). Thus, it would be obvious to select a desired target region as taught by Talgorn and within said target region, further select a target location as taught by Fiedler because such a target location selection may improve patient outcomes by preventing the medical device from being attached in an undesirable location. Furthermore, it would be obvious to configure the guidance of Talgorn to provide guidance to the user to place the medical device in the target location because such guidance would ensure proper device placement and improve patient outcomes.
Regarding claim 7, Talgorn in view of Fiedler teaches the method of claim 1. Modified Talgorn fails to further teach the method wherein determining the target location comprises identifying one or more unsuitable areas of the patient's body based on the detected one or more characteristics.
Fiedler teaches the use of optical sensors to capture images of the body and identify references points and injection sites. The system logs the injection sites including their time and may indicate that an injection site is not usable until a certain amount of time has passed or other criteria are met (Paragraphs 0184-0191). The system may determine where valid injection sites are and recommend a target site location based on a number of configurable criteria including proximity from last injection site, amount of time that has passed since the previous injection at that site, degree of lipodystrophy, and other factors (Paragraphs 0192-0194).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to implement the site recommendation and identification of valid sites based on how much time has elapsed since the patient was last administer an injection at a given site as taught by Fielder into the method of modified Talgorn because Fielder teaches that a certain amount of time may need to pass before an injection site may be used again to prevent lipodystrophy (lipos) (Fiedler: Paragraph 0082) and incorporating this site tracking to identify invalid and valid sites would improve user outcomes when using the method of modified Talgorn by helping prevent lipos.
Regarding claim 8, Talgorn in view of Fiedler teaches the method of claim 7. Modified Talgorn further teaches the method, wherein (i) the one or more unsuitable areas of the patient's body comprise areas with one or more of inflammation, infection, eczema, cancer, and psoriasis and/or (ii) identifying the one or more unsuitable areas is based in part on how much time has elapsed since the patient last administered an injection from any medical device, and/or (iii) identifying the one or more unsuitable areas is based in part on orientation limitations of the medical device.
Fiedler teaches the use of optical sensors to capture images of the body and identify references points and injection sites. The system logs the injection sites including their time and may indicate that an injection site is not usable until a certain amount of time has passed or other criteria are met (Paragraphs 0184-0191). The system may determine where valid injection sites are and recommend a target site location based on a number of configurable criteria including proximity from last injection site, amount of time that has passed since the previous injection at that site, degree of lipodystrophy, and other factors (Paragraphs 0192-0194).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to implement the site recommendation and identification of valid sites based on how much time has elapsed since the patient was last administer an injection at a given site as taught by Fielder into the method of modified Talgorn because Fielder teaches that a certain amount of time may need to pass before an injection site may be used again to prevent lipodystrophy (lipos) (Fiedler: Paragraph 0082) and incorporating this site tracking and rotation would improve user outcomes when using the method of modified Talgorn by helping prevent lipos.
Regarding claim 11, Talgorn in view of Fiedler teaches the method of claim 7. Modified Talgorn further teaches the method, wherein determining the target location comprises generating a photogrammetric model of the patient and identifying the one or more unsuitable areas is based in part on the photogrammetric model.
Fiedler teaches the generation of a coordinate system of the patient using reference points identified in images and determines the suitable reference sites based on the coordinates of the body area as imaged (Paragraphs 0184-0194). The generation of a coordinate system using the captured images of the patient is considered to teach the generation of a “photogrammetric model”.
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to implement the generating of a coordinate system on the captured images and using the coordinate system to determine suitable injection sites as taught by Fielder into the method of modified Talgorn because Fielder teaches that a certain amount of time may need to pass before an injection site may be used again to prevent lipodystrophy (Fiedler: Paragraph 0082) and incorporating this site tracking and rotation would improve user outcomes when using the method of modified Talgorn by helping prevent lipos.
Regarding claim 14, Talgorn in view of Fiedler teaches the method of claim 7. Modified Talgorn further teaches the method, wherein generating the photogrammetric model comprises obtaining a plurality of body reference keypoints in near real-time based on the one or more images (Paragraphs 0066-0067: the detection of various body markers, or body reference keypoints, including detecting markers during removal of a previously placed wearable device which is considered to be “near real-time”).
Regarding claim 15, Talgorn in view of Fiedler teaches the method of claim 4. Modified Talgorn fails to further teach the method, wherein determining the target location comprises identifying a target region on the patient's body based on the body reference keypoints and identifying the target location within the target region based on one or more unsuitable areas.
Talgorn does teach the identification of a plurality of body reference keypoints (Paragraphs 0066-0067) and the identification of target regions (Paragraphs 0053 and 0062) but fails to explicitly recite that the identification of the target regions is based on the body reference keypoints or identifying the target location within the target region based on the one or more unsuitable areas.
Fiedler teaches the use of reference points in selected body areas to generate a cartesian coordinate grid. Fiedler further teaches that the target injection location may be determined based on which areas are valid injection sites and which are not yet able to be used again (Paragraphs 0184-0194). Thus, the body areas have corresponding reference features and the target location is determined with respect to unsuitable locations and is considered “based on” the unsuitable areas.
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to configure the method of modified Talgorn to identify target regions based on the reference features within the respective regions and to determine the target location based on the unsuitable areas as taught by Fiedler because each body region may include unique identifiers which may help in accurately determining the body region and Fiedler teaches that certain areas are not valid for receiving injections until a certain amount of time has passed and thus the selection of a target location must consider the invalid sites in order to prevent undesirable outcomes.
Regarding claim 16, Talgorn in view of Fiedler teaches the method of claim 15. Modified Talgorn fails to further teach the method wherein determining the target location comprises: mapping the target region to the photogrammetric model of the patient; un-mapping the one or more unsuitable areas from the mapped photogrammetric model; and identifying the target location within the mapped photogrammetric model.
Fiedler teaches the generation of a coordinate system on the body area which includes the coordinates of past injection sites. The target location may be determined with respect to the past injection sites and/or body areas to be avoided with respect to corresponding time periods that must pass before an injection site can be used again. The method may then automatically determine a suggested target site based on the valid injection sites (Paragraphs 0184-0194). The generation of the coordinate system is considered to teach the “mapping” of the target region, the avoidance of past injection sites is considered to teach “un-mapping” those regions as they are indicated as invalid injection sites, the target injection region is then determined from the valid, or mapped injection locations.
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to incorporate the mapping of the target region and determining the target location based on valid injection sites as taught by Fiedler into the method of modified Talgorn because Fielder teaches that a certain amount of time may need to pass before an injection site may be used again to prevent lipodystrophy (Fiedler: Paragraph 0082) and incorporating this site tracking and selection would improve user outcomes when using the method of modified Talgorn by helping prevent lipos.
Regarding claims 23-25, Talgorn in view of Fiedler teaches the method of claim 1. Modified Talgorn further teaches the method wherein detecting the location of the medical device comprises identifying one or more device reference keypoints based on the one or more images (Paragraph 0050: the wearable device including markers identified by the camera for tracking); wherein generating guidance comprises generating a 2-dimensional (2D) positioning vector based on the one or more device reference keypoints and the target location, and wherein the one or more user interface objects comprise a user interface object corresponding to the 2D positioning vector on the computing device (Paragraphs 0057-0060: the guidance is generated based on the position of the wearable device relative to the body part and the guidance may include arrows, or 2D vectors); and wherein generating guidance comprises generating a 3-dimensional (3D) positioning rotation angle based on the one or more device reference keypoints and the target location, and wherein the one or more user interface objects comprise a user interface object corresponding to the 3D positioning rotation angle on the computing device. (Paragraphs 0057-0060 and 0073: the guidance may be provided in three dimensions and may include angle guidance)
Regarding claim 34 Talgorn discloses an interactive patient guidance system (Abstract) comprising:
a medical device (Fig. 4 reference 404; Paragraph 0050: the wearable device);
a computing device (Fig. 4 reference 402; Paragraph 0050: the apparatus) comprising:
one or more cameras (Fig. 4 reference 104; Paragraph 0050: the camera);
one or more indicators (Paragraphs 0058-0059: the guidance elements such as lights and the screen); and
one or more processors (Paragraphs 0049 and 0051-0052: the processor of the apparatus) configured to run instructions to:
acquire one or more images of a patient via the one or more cameras (Paragraphs 0052 and 0062: at least one acquired image from one or more cameras);
detect one or more characteristics of the patient's body in the one or more images (Paragraphs 0053 and 0062: the images are analyzed to recognize body parts; Paragraph 0066-0067: the method may further involve identifying markers such as skin fold, pores, birthmarks, and other body markers including markers that are acquired during removal of a previously placed device);
determine a target region for placing the medical device on the patient's body based on the detected one or more characteristics (Paragraphs 0053 and 0062: The recognized body parts are used to identify the body part at which to place the device);
detect a location of the medical device (Paragraph 0060: detecting the location of the wearable device);
generate guidance for moving the medical device from the detected location of the medical device to the determined target region (Paragraphs 0057-0060: the guidance to position the wearable device in relation to the identified body part is provided and must therefore also be generated); and
provide the guidance to the patient via the one or more indicators on the computing device (Paragraphs 0057-0060: the guidance is provided to the user through the device itself and/or through an external device such as a smart mirror).
Talgorn fails to further disclose the system including determining a particular target location and guiding the device to the target location.
Fiedler teaches the use of optical sensors to capture images of the body and identify references points and injection sites. The system logs the injection sites including their time and may indicate that an injection site is not usable until a certain amount of time has passed or other criteria are met (Paragraphs 0184-0191). The system may determine where valid injection sites are and recommend a target site location based on a number of configurable criteria including proximity from last injection site, amount of time that has passed since the previous injection at that site, degree of lipodystrophy, and other factors (Paragraphs 0192-0194).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to implement the selection of a particular target location as taught by Fiedler because Fielder teaches that injection sites may need to be rotated to prevent adverse effects and further contemplates that other criteria may need to be met before a particular site may be used again (Fiedler: Paragraph 0191). Thus, it would be obvious to select a desired target region as taught by Talgorn and within said target region, further select a target location as taught by Fiedler because such a target location selection may improve patient outcomes by preventing the medical device from being attached in an undesirable location. Furthermore, it would be obvious to configure the guidance of Talgorn to provide guidance to the user to place the medical device in the target location because such guidance would ensure proper device placement and improve patient outcomes.
Regarding claim 40, Talgorn in view of Fiedler teaches the system of claim 34. Modified Talgorn fails to further disclose the system wherein determining the target location comprises identifying one or more unsuitable areas of the patient's body based on the detected one or more characteristics.
Fiedler teaches the use of optical sensors to capture images of the body and identify references points and injection sites. The system logs the injection sites including their time and may indicate that an injection site is not usable until a certain amount of time has passed or other criteria are met (Paragraphs 0184-0191). The system may determine where valid injection sites are and recommend a target site location based on a number of configurable criteria including proximity from last injection site, amount of time that has passed since the previous injection at that site, degree of lipodystrophy, and other factors (Paragraphs 0192-0194).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to implement the site recommendation and identification of valid sites based on how much time has elapsed since the patient was last administer an injection at a given site as taught by Fielder into the system of modified Talgorn because Fielder teaches that a certain amount of time may need to pass before an injection site may be used again to prevent lipodystrophy (lipos) (Fiedler: Paragraph 0082) and incorporating this site tracking to identify invalid and valid sites would improve user outcomes when using the method of modified Talgorn by helping prevent lipos.
Regarding claim 41, Talgorn in view of Fiedler teaches the system of claim 40. Modified Talgorn fails to further disclose the system, wherein (i) the one or more unsuitable areas of the patient's body comprise areas with one or more of inflammation, infection, eczema, cancer, and psoriasis, and/or (ii) identifying the one or more unsuitable areas is based in part on how much time has elapsed since the patient last administered the medical device, and/or (iii) identifying the one or more unsuitable areas is based in part on orientation limitations of the medical device.
Fiedler teaches the use of optical sensors to capture images of the body and identify references points and injection sites. The system logs the injection sites including their time and may indicate that an injection site is not usable until a certain amount of time has passed or other criteria are met (Paragraphs 0184-0191). The system may determine where valid injection sites are and recommend a target site location based on a number of configurable criteria including proximity from last injection site, amount of time that has passed since the previous injection at that site, degree of lipodystrophy, and other factors (Paragraphs 0192-0194).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to implement the site recommendation and identification of valid sites based on how much time has elapsed since the patient was last administer an injection at a given site as taught by Fielder into the system of modified Talgorn because Fielder teaches that a certain amount of time may need to pass before an injection site may be used again to prevent lipodystrophy (lipos) (Fiedler: Paragraph 0082) and incorporating this site tracking and rotation would improve user outcomes when using the system of modified Talgorn by helping prevent lipos.
Regarding claim 44, Talgorn in view of Fiedler teaches the system of claim 40. Modified Talgorn fails to further disclose the system wherein determining the target location comprises generating a photogrammetric model of the patient and identifying the one or more unsuitable areas is based in part on the photogrammetric model.
Fiedler teaches the generation of a coordinate system of the patient using reference points identified in images and determines the suitable reference sites based on the coordinates of the body area as imaged (Paragraphs 0184-0194). The generation of a coordinate system using the captured images of the patient is considered to teach the generation of a “photogrammetric model”.
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to implement the generating of a coordinate system on the captured images and using the coordinate system to determine suitable injection sites as taught by Fielder into the system of modified Talgorn because Fielder teaches that a certain amount of time may need to pass before an injection site may be used again to prevent lipodystrophy (Fiedler: Paragraph 0082) and incorporating this site tracking and rotation would improve user outcomes when using the system of modified Talgorn by helping prevent lipos.
Claims 2 and 35 are rejected under 35 U.S.C. 103 as being unpatentable over Talgorn US Patent Application Publication Number US 20200121249 A1 hereinafter Talgorn in view of Fiedler US Patent Application Publication Number US 20190076604 A1 hereinafter Fiedler as applied to claims 1 and 34 above and further in view of Yoo US Patent Application Publication Number US 20240008839 A1 hereinafter Yoo.
Regarding claim 2, Talgorn in view of Fiedler teaches the method of claim 1. Modified Talgorn fails to further disclose the method, wherein the computing device comprises a mirror.
In particular, Talgorn teaches the use of a smart mirror (Paragraphs 0038, 0041, and 0058) but Talgorn teaches that the processor which executes the above-described algorithm is located within the “apparatus” and the smart mirror is described as an external tool being controlled by the apparatus rather than executing the process itself.
Yoo teaches a device which guides a proper posture correction of an ultrasound scanner with an augmented reality that delivers feedback control commands to a patient who self-diagnoses through a smart mirror and enables a patient to self-diagnose breast cancer by an artificial intelligence neural network that has been learned by thermographic and ultrasound images (Abstract). Thus, Yoo falls within the same field of endeavor as Applicant’s invention.
Yoo teaches a smart mirror including a controller which provides guidance on positioning a medical device (Paragraphs 0166 and 0200-0203).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to configure the method of Talgorn such that the smart mirror carries out the method and provides guidance to the apparatus of Talgorn as taught by Yoo because such a configuration may reduce the power and processing requirements of the apparatus of Talgorn since the object recognition and guidance generation which may be computationally intensive are carried out by the mirror which may have reduced size and power consumption constraints. Additionally, such a configuration is considered a mere rearrangement of parts since the processor of the apparatus of Talgorn is merely being relocated to the smart mirror and no functionality has been changed.
Regarding claim 35, Talgorn in view of Fiedler teaches the system of claim 34. Modified Talgorn fails to further disclose the system, wherein the computing device comprises a mirror.
In particular, Talgorn teaches the use of a smart mirror (Paragraphs 0038, 0041, and 0058) but Talgorn teaches that the processor which executes the above-described algorithm is located within the “apparatus” and the smart mirror is described as an external tool being controlled by the apparatus rather than executing the process itself.
Yoo teaches a smart mirror including a controller which provides guidance on positioning a medical device (Paragraphs 0166 and 0200-0203).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to configure the method of Talgorn such that the smart mirror carries out the method and provides guidance to the apparatus of Talgorn as taught by Yoo because such a configuration may reduce the power and processing requirements of the apparatus of Talgorn since the object recognition and guidance generation which may be computationally intensive are carried out by the mirror which may have reduced size and power consumption constraints. Additionally, such a configuration is considered a mere rearrangement of parts since the processor of the apparatus of Talgorn is merely being relocated to the smart mirror and no functionality has been changed.
Claims 19-20 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Talgorn US Patent Application Publication Number US 20200121249 A1 hereinafter Talgorn in view of Fiedler US Patent Application Publication Number US 20190076604 A1 hereinafter Fiedler as applied to claims 1 and 14 above and further in view of Fazeli US Patent Number US 12004871 B1 hereinafter Fazeli
Regarding claims 19-20, Talgorn in view of Fiedler teaches the method of claim 14. Modified Talgorn fails to further teach the method wherein obtaining the plurality of estimated body reference keypoints comprises processing the one or more images using a machine-learning model for estimating pose, or wherein the machine-learning model is a convolutional neural network model.
Fazeli teaches the generation of a 3D model of a user from 2D images (Abstract). Thus, Fazeli is reasonably pertinent to the problem at hand.
Fazeli teaches that Convolutional neural networks (CNN) may be used to process image features and generate body parameters including pose angles, body shape, joint locations, and other parameters (Col 28 line 37 – Col 29 line 12).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to implement the CNN of Fazeli into the method of modified Talgorn to perform the reference keypoint detection using the captured images because Fazeli teaches that CNN are well suited to image processing and feature extraction and performing the keypoint extraction using a CNN may improve the accuracy and efficiency of the algorithm since neural networks are highly adaptable and configurable to a particular use.
Regarding claim 27 Talgorn in view of Fiedler teaches the method of claim 1. Modified Talgorn further teaches the method wherein detecting one or more characteristics of the patient comprises processing the one or more images for detecting dermatological conditions (Paragraph 0066: the detected features may include skin spots, skin tone, birthmarks, and other dermatological conditions).
Modified Talgorn fails to further disclose the detection being carried out using a machine-learning model.
Fazeli teaches that Convolutional neural networks (CNN) may be used to process image features and generate body parameters including pose angles, body shape, joint locations, and other parameters (Col 28 line 37 – Col 29 line 12).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to implement the CNN of Fazeli into the method of modified Talgorn to perform the dermatological marker detection using the captured images because Fazeli teaches that CNN are well suited to image processing and feature extraction and performing the keypoint extraction using a CNN may improve the accuracy and efficiency of the algorithm since neural networks are highly adaptable and configurable to a particular use.
Claim 67 is rejected under 35 U.S.C. 103 as being unpatentable over Talgorn US Patent Application Publication Number US 20200121249 A1 hereinafter Talgorn in view of Yoo US Patent Application Publication Number US 20240008839 A1 hereinafter Yoo.
Regarding claim 67 Talgorn discloses a medical device (Abstract) comprising:
one or more indicators (Paragraphs 0058-0059: the guidance elements such as lights and the screen); and
one or more processors (Paragraphs 0049 and 0051-0052: the processor of the apparatus);
wherein the medical device is communicatively connected to an external processor (Paragraphs 0040-0042 and 0058: the apparatus may be in communication with external devices such as smart devices which include processors) and the one or more processors of the medical device are configured to:
provide the guidance to the patient via the one or more indicators of the medical device (Paragraphs 0058-0059: the guidance may be provided to the user via the apparatus itself and/or external devices including lights or other indicators on a display of the apparatus).
Talgorn fails to further disclose the apparatus configured to: receive one or more instructions from the external processor for providing guidance to a patient for placing the medical device on the patient's body.
Yoo teaches a smart mirror including a controller which provides guidance on positioning a medical device (Paragraphs 0166 and 0200-0203).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of the invention to configure the device of Talgorn such that the external smart mirror carries out the method and provides guidance to the apparatus of Talgorn as taught by Yoo because such a configuration may reduce the power and processing requirements of the apparatus of Talgorn since the object recognition and guidance generation which may be computationally intensive are carried out by the mirror which may have reduced size and power consumption constraints. Additionally, such a configuration is considered a mere rearrangement of parts since the processor of the apparatus of Talgorn is merely being relocated to the smart mirror and no functionality has been changed.
Claims 12-13 and 21-22 have not been rejected over the prior art but are nonetheless rejected in light of the above presented 35 USC 101 rejections.
In regards to claim 12, none of the prior art of record teaches or reasonably suggests “determining three dimensional normal surface vectors of the photogrammetric model, and the one or more unsuitable areas comprise areas wherein the three dimensional normal surface vectors are incompatible with one or more orientation limitations of the medical device” in combination with the other claimed elements
In regards to claim 13, none of the prior art of record teaches or reasonably suggests “wherein identifying the one or more unsuitable areas comprises determining three dimensional gradient surface vectors of the photogrammetric model, and the one or more unsuitable areas comprise areas wherein the three dimensional gradient surface vectors are incompatible with one or more orientation limitations of the medical device” in combination with the other claimed elements
In regards to claim 21, none of the prior art of record teaches or reasonably suggests “inferring the location of the medical device based on the custom set of anthropometric ratios for the patient” in combination with the other claimed elements.
Claim 22 depends from claim 21 and is thus not rejected on the same grounds as claim 21.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW ERIC OGLES whose telephone number is (571)272-7313. The examiner can normally be reached M-F 8:00AM - 5:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Sims can be reached on Monday-Friday from 9:00AM – 4:00PM at (571) 272 – 7540. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW ERIC OGLES/Examiner, Art Unit 3791
