DETAILED ACTION
Applicant has canceled claims 2 and 6. Claims 1, 3-5, and 7-20 are pending and hereby under examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 10 is objected to because of the following informalities:
Claim 10, line 3, “the posture transition” should recite “a posture transition”.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“radar system for generating” first recited in claim 20;
“first processor adapted to process” first recited in claim 20;
“second processor adapted to process” first recited in claim 20;
“tracker adapted to track” first recited in claim 20;
“identifier adapted to identify” first recited in claim 20; and
“analyzer adapted to analyze” first recited in claim 20.
The identified structure for the corresponding claim limitations are as follows:
“radar system” is identified as “The radar device 10 comprises a transmitter 11 and three receivers 12, 13, 14. The transmitter 11 and the three receivers 12, 13, 14 are arranged within a housing of the radar device 10 in a certain distance from each other such that differences with respect to incoming signal phase are caused” (Page 16, line 24 – Page 17, line 4).
“first processor” is identified as
“1. The radar data is received by the processor from the transceivers. The data is organized in a matrix, where the columns correspond to separate, consecutive chirps. The lines of a given column represent the samples of the given chirp.
2. A matrix of a symmetric window function is generated, where the window length corresponds to the samples per chirp (number of lines) and the number of chirps (number of columns).
3. For each transceiver, a (1-dimensional) range Doppler map is calculated based on an
average of a succession of radar signals (to reduce noise), windowed by the window function, applying a 2-dimensional Fourier transformation and shifting zero Doppler to the middle of the x axis.
4. Now, the phase difference may be obtained from two range Doppler maps MRD1, MRD2, by calculating MRD,1 · MRD,2. This step may be repeated or generalized to more than two transceivers.
5. From this product or these products, respectively, the angles of arrival may be calculated from the phases of the matrix elements.
From the range Doppler maps, point clouds are generated by a processing module 31, consisting of all points where a movement was detected by the radar system. Each of the movement points is characterized by its three-dimensional position and its three-dimensional velocity vector” (Page 18, lines 1-19).
“second processor” is identified as “Now, a dedicated, density-based clustering algorithm is run on a clustering module 33, using unsupervised learning methods to group the movement points of each frame according to their characteristics, based on the information contained in the feature vectors. The clustering algorithm is also configured to eliminate noise by detecting not- clustered points ("outliers") and irregular groups. A cluster is a group of movement points which belong to the same moving object. It can have an arbitrary form and size, depending on the detected movements and their distribution.
Two clustering processes are run in parallel:
- For the clustering of entire people, no further parameters are added to the feature vector, as different body parts can move in different directions and have different levels of reflected energy.
- For the clustering of specific body parts (e.g. chest, extremities) the velocity vector of the movement is added to every point's feature vector. This is important as all the detected movements of a specific part are expected to move in more or less the same15 diction with more or less the same velocity. This creates a high-dimensional clustering space which results in smaller, more accurate clusters.” (Page 19, lines 1-16).
“tracker” is identified as “each cluster is tracked from frame to frame by a tracking module 35. The tracking algorithm calculates for every cluster in every frame the probability that it is the same cluster for a previous frame. The probability calculation takes into account different characteristics of the group – current position, path so far (direction and velocity) and points’ reflection characteristics. Each tracked group is then identified as an “object” and is given a unique identifier (ID)” (Page 19, lines 20-25) and “In a further step, the remaining, i.e. human objects are classified as static or dynamic. For that purpose, the positions and velocities of these remaining objects are studied during a time interval. If the overall movement is low and/or the position of the object is restricted to a certain location, the object is classified as static, otherwise it is classified as dynamic” (Page 29, lines 3-6).
“identifier” is identified as “the size and form of the respective cluster: this indicates the volume of the person’s body or body part which is producing movements; the density and distribution of the movement points in the respective cluster: where the density of the movement-points is higher, more movement is being generated; therefore, changes in the distribution indicate the amplitude of movements in certain areas of the cluster, i.e. of different body parts; and the velocity and direction of movement for each point in the respective cluster: this gives an indication of the changes in movements in a higher resolution” (Page 20, lines 12-19).
“analyzer” is identified as “if the object exhibits substantial movement with generally uniform distribution in the point cloud, the shape thereof represents more or less the shape of the person. The classification of the basic three postures ‘lying’, ‘sitting’, or ‘standing’ is done in a classification module 41, using a machine learning algorithm, trained on labeled samples, based on the entire point cloud representing the patient. If there is less movement, the determination of the posture is based on the tracked groups representing body parts. In order to do so, first the body part group representing the chest is identified in identification module 43, this should always be possible, because the chest of a living person always generates movement” (Page 21, lines 1-9).
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 3-5, and 7-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 1 and 20, the claim recites the step of “taking into account for the clustering of body parts the velocity vectors of the points of the point cloud”. It is unclear what “taking into account” requires for the clustering step. Are the velocity vectors used in an algorithm or equation when clustering? Do the velocity vectors have a weight associated with them during the clustering step. For examination purposes, the claim will be interpreted such that the velocity vectors must be used in some way during the clustering step. Claims 3-5 and 7-19 are also rejected due to their dependence on claim 1.
Regarding claim 18, it is unclear if the single values are collected over a first sample period or a second sample period. Does the claim require the single values to be collect twice over two separate sample periods, or once over two sample periods? If there are two separate collections of the single values, are the single values collected over the first sample period separate and distinct from the second sample period, or are they the same? For examination purposes, claim 18 will be interpreted such that the single values collected over the second sample period refer to the same or similar single values collected over the first sample period but over a separate time period.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Analysis of independent claims 1-20:
Step 1 of the subject matter eligibility test (see MPEP 2106.03).
Claim 20 is directed to a system, which describes one of the four statutory categories of patentable subject matter, i.e., a machine. Claim 1 is directed to a computer implemented method, which describes one of the four statutory categories of patentable subject matter, i.e., a method. Therefore, further consideration is necessary regarding claims.
Step 2A of the subject matter eligibility test (see MPEP 2106.04).
Prong One: Claims 1 and 20 recite an abstract idea. In particular, the claims generally recite
the following:
processing the measurement data to obtain three dimensional point cloud data, a position and local data being assigned to each of the points of the point cloud data, the local data comprising velocity vectors of the points;
clustering the point cloud data to obtain clustered data representing the human being by a plurality of clusters, taking into account for the clustering of body parts the velocity vectors of the points of the point cloud;
calculating probability for every cluster in every frame that the cluster relates to the same detected object as a certain cluster in the previous frame,
for defining the objects, analyzing data obtained from tracking the clusters, covering a time interval, to separate static from dynamic objects;
identify body parts of the human being and positional relationships among them based on the plurality of objects, wherein the identified body parts comprise the head and/or limbs; and
analyze the positional relationships to determine the posture of the human being.
These elements recited in claims 1 and 20 are drawn to an abstract idea since they are directed towards mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP § 2106.04(a)(2), subsection I) and mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III).
“processing the measurement data to obtain three dimensional point cloud data, a position and local data being assigned to each of the points of the point cloud data, the local data comprising velocity vectors of the points” is drawn to an abstract idea since it is a mental process that can be practically performed in the human mind, with the aid of pen and paper or a generic computer. A person of ordinary skill in the art could reasonably review radar measurements to calculate positional and velocity data. There is nothing to suggest an undue level of complexity in “processing the measurement data to obtain three dimensional point cloud data, a position and local data being assigned to each of the points of the point cloud data, the local data comprising velocity vectors of the points”.
“clustering the point cloud data to obtain clustered data representing the human being by a plurality of clusters, taking into account for the clustering of body parts the velocity vectors of the points of the point cloud” is drawn to an abstract idea since it is a mental process that can be practically performed in the human mind, with the aid of pen and paper or a generic computer. A person of ordinary skill in the art could reasonably take positional and velocity data, and cluster the data into a plurality of clusters. There is nothing to suggest an undue level of complexity in “clustering the point cloud data to obtain clustered data representing the human being by a plurality of clusters, taking into account for the clustering of body parts the velocity vectors of the points of the point cloud”.
“calculating probability for every cluster in every frame that the cluster relates to the same detected object as a certain cluster in the previous frame” is drawn to a mathematical concept as calculating a probability is a common mathematical formula.
“for defining the objects, analyzing data obtained from tracking the clusters, covering a time interval, to separate static from dynamic objects” is drawn to an abstract idea since it is a mental process that can be practically performed in the human mind, with the aid of pen and paper or a generic computer. A person of ordinary skill in the art could reasonably review the clustered data to differentiate a static object from a moving object. There is nothing to suggest an undue level of complexity in “for defining the objects, analyzing data obtained from tracking the clusters, covering a time interval, to separate static from dynamic objects”.
“identify body parts of the human being and positional relationships among them based on the plurality of objects, wherein the identified body parts comprise the head and/or limbs” is drawn to an abstract idea since it is a mental process that can be practically performed in the human mind, with the aid of pen and paper or a generic computer. A person of ordinary skill in the art could reasonably view the clustered data to identify body parts of the human. There is nothing to suggest an undue level of complexity in “identify body parts of the human being and positional relationships among them based on the plurality of objects, wherein the identified body parts comprise the head and/or limbs”.
“analyze the positional relationships to determine the posture of the human being” is drawn to an abstract idea since it is a mental process that can be practically performed in the human mind, with the aid of pen and paper or a generic computer. A person of ordinary skill in the art could reasonably view the identified body parts to determine how the human is postured. There is nothing to suggest an undue level of complexity in “analyze the positional relationships to determine the posture of the human being”.
Prong Two: Claims 1 and 20 do not recite additional elements that integrate the exception into a practical application. Therefore, the claims are "directed to" the abstract idea. The additional elements merely:
Recite the words "apply it" or an equivalent with the judicial exception, or include instructions to implement the abstract idea on a computer, or merely use the computer as a tool to perform the abstract idea (e.g., “a first processor” (claim 20), "a second processor" (claim 20), “a tracker” (claim 20), “an identifier” (claim 20), and “an analyzer” (claim 20) and
Add insignificant extra-solution activity (the pre-solution activity of: using generic data gathering components (e.g., "obtaining measurement data from at least two dimensional radar measurements" (claim 1) and "a radar system for generating measurement data from at least two dimensional radar measurements from a space accommodating the human being" (claim 20))).
As a whole, the additional elements merely serve to gather information to be used by the abstract idea, while generically implementing it on a computer. There is no practical application because the abstract idea is not applied, relied on, or used in a meaningful way. The processing performed remains in the abstract realm, i.e., the result is not used for a treatment. No improvement to the technology is evident. Therefore, the additional elements, alone or in combination, do not integrate the abstract idea into a practical application.
Step 2B of the subject matter eligibility test (see MPEP 2106.05).
Claims 1 and 20 do not include additional elements, alone or in combination, that are sufficient to amount to significantly more than the judicial exception (i.e., an inventive concept) for the same reasons as described above. E.g., all elements are directed to implementing the abstract ideas on generic processing components, the pre-solution activity of using generic data-gathering components, and generic post-solution activities, which merely facilitate the abstract idea.
Per the Berkheimer requirement, the additional elements are well-understood, routine, and conventional. For example, “a radar system” as disclosed in the Applicant’s specification, “Preferably, the radar system comprises units featuring senders and receivers arranged in a common unit. The number of senders and receivers is chosen depending on the employed electromagnetic waves and the desired information” (Page 3, lines 16-18) and “In particular, the radar measurements are obtained from a wideband radar system, i. e. a radar system having a bandwidth of at least 100 MHz. In a preferred embodiment, the wideband radar system is an ultra-wideband (UWB) radar system. Such systems have a bandwidth exceeding the lesser of 500 MHz or 20% of the arithmetic center frequency. Preferably, the center frequency of the wideband or ultra-wideband radar system is in the range of 2 - 75 GHz. Advantageously, the range resolution of the radar system is 5 mm or better. Wideband or ultra-wideband radar systems as well as center frequencies in the given range allow for high resolution, sufficient penetration of objects such as bed sheets and sufficient reflection from the body of the patient to be monitored. In a preferred embodiment, the wideband radar system is a MIMO (multiple-input multiple- output) radar system, featuring a number of transmitting antennas sending different transmitting signals (in particular orthogonal or intermittent signals) as well as a number of receiving antennas receiving signals from different transmitting antennas” (Page 4, lines 6-19).
Further, “processor”, “tracker”, “identifier”, and “analyzer” do not qualify as significantly more because this limitation is simply appending well understood, routine and conventional activities previously known in the industry, specified at a high level of generality, to the judicial exception, e.g., a claim to an abstract idea requiring no more than a generic computer to perform generic computer functions that are well-understood, routine and conventional activities previously known in the industry (see Electric Power Group, 830 F.3d 1350 (Fed. Cir. 2016); Alice Corp. v. CLS Bank Int'/, 110 USPQ2d 1976 (2014)) and/or a claim to an abstract idea requiring no more than being stored on a computer readable medium which is a well understood, routine and conventional activity previously known in the industry (see Electric PowerGroup, 830 F.3d 1350 (Fed. Cir. 2016); Alice Corp. v. CLS Bank Int'/, 110 USPQ2d 1976 (2014); SAP Am. v. lnvestPic, 890 F.3d 1016 (Fed. Circ. 2018)).
In view of the above, the additional elements individually do not integrate the exception into a practical application and do not amount to significantly more than the above-judicial exception (the abstract idea). Looking at the limitations as an ordered combination (that is, as a whole) adds nothing that is not already present when looking at the elements taking individually. There is no indication that the combination of elements improves the functioning of a computer, for example, or improves any other technology. There is no indication that the combination of elements permits automation of specific tasks that previously could not be automated. There is no indication that the combination of elements include a particular solution to a computer-based problem or a particular way to achieve a desired computer-based outcome. Rather, the collective functions of the claimed invention merely provide conventional computer implementation, i.e., the computer is simply a tool to perform the process.
Analysis of the dependent claims:
Claims 3-5 and 7-19 depend from the independent claims. Dependent claims 3-5 and 7-19 merely further define the abstract idea and are, therefore, directed to an abstract idea for similar reasons: they merely
Further describe the abstract idea (“wherein the local data assigned to each of the points of the point cloud data comprises a reflection magnitude” (claim 3), “wherein the local data assigned to each of the points of the point cloud data comprises a signal-to-noise ratio of an underlying radar measurement and/or a measure for a confidence of the position and/or the local data” (claim 4), “wherein the objects are classified to identify non-human objects and wherein objects representing non-human objects are excluded from the identification of the body parts of the human being and the positional relationships among them” (claim 5), “wherein the static objects located in predefined zones are identified as patients” (claim 7), “wherein an activity level is determined from the point cloud data and wherein an overall shape defined by the points of the plurality of clusters representing the human being is analyzed to determine the posture if the activity level exceeds a threshold” (claim 8), “mapping further points of the point cloud data, obtained from further measurements after determining the posture, to the determined posture; and identifying points of the mapped further points lying outside of the determined posture in order to identify a posture transition” (claim 9), “wherein a range Doppler map is obtained from the at least two dimensional radar measurements and wherein information on obtained clusters, defined objects and/or identified body parts is used to identify regions of interest in the range Doppler map, the regions of interest of the range Doppler map being analyzed in order to do at least one of the following: identify valid radar measurements to be considered for obtaining the at least two dimensional point cloud data; select radar measurements to be considered for extracting vital sign information; or clustering micro Doppler radar measurements in one of the regions of interest or starting from a region of interest” (claim 12), “further comprising the steps of identifying an object that represents a chest of a human being, based on characteristic features relating to breathing movements and/or heartbeat; and determining a positional relationship of other objects to the identified chest object to identify the body parts of the human being and the positional relationships among them” (claim 13), “wherein a movement activity level is determined from an amplitude and/or phase of the measurement data, wherein the activity level is classified into at least one high activity level class and into at least one low activity level class and wherein the amplitude and/or phase of the measurement data relating to an activity level classified in the at least one low activity level class is analyzed to identify the position of the chest region” (claim 14), “wherein each object is classified according to its geometrical shape into a torso candidate class or a non- torso candidate class” (claim 15), “wherein for each of the objects a single value is calculated for all the points of the point cloud assigned to the respective cluster, collected during a first sample period, the single value representing an activity level in the respective cluster” (claim 16), “wherein the single value is calculated from a number of points collected and absolute velocities of each of the points” (claim 17), “wherein the single values are collected over a second sample period” (claim 18), and “wherein a frequency analysis is performed over the collected single values and classifying each of the objects into a chest candidate class or a non-chest candidate class” (claim 19)), and
Further describe the post-solution activity (“wherein a risk for the human being experiencing a fall incident is determined based on the posture and/or the posture transition and wherein an alarm condition is set for the human being if the risk exceeds a threshold” (claim 10) and “wherein the alarm condition for the human being is reset if it is determined that the human being is approached by a dynamic object representing a caregiver” (claim 11), etc.).
Taken alone or in combination, the additional elements do not integrate the judicial exception into a practical application at least because the abstract idea is not applied, relied on, or used in a meaningful way. The additional elements do not add anything significantly more than the abstract idea. The collective functions of the additional elements merely provide computer/electronic implementation and processing, and no additional elements beyond those of the abstract idea. There is no indication that the combination of elements permits automation of specific tasks that previously could not be automated. There is no indication that the combination of elements improves the functioning of a computer, output device, improves technology other than the technical field of the claimed invention, etc. The result of the abstract idea does not cause the computing device and/or application to perform different. The result of the abstract idea does not cause output of the user-accessible output.
Therefore, claims 1, 3-5, and 7-20 are rejected as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3-5, 7-9, 12, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Cui et. al. (“Real-Time Short-Range Human Posture Estimation Using mmWave Radars and Neural Networks”), hereinafter Cui, and Moon (KR 20220071622).
Regarding claims 1, 7-8, and 10, Cui teaches a method for determining a posture of a human being, the method comprising the following steps:
obtaining measurement data from at least two dimensional radar measurements (Fig. 2 below, radar 1 and radar 2; Page 536, section A. Radar Characterisation, “The arrangement of the transmitters enables the radar to capture both azimuth and elevation information”);
PNG
media_image1.png
183
220
media_image1.png
Greyscale
processing the measurement data to obtain three dimensional point cloud data, a position and local data being assigned to each of the points of the point cloud data, the local data comprising velocity vectors of the points (Page 537, section B. Data Collection and Pre-Processing, “The radar outputs a 3D point cloud in arbitrary sizes, representing the geometric information about any subject in front of it”; Page 536, section A. Radar Characterisation, “The resulting point cloud will encode the x-y-z coordinates and velocity of the object”);
clustering the point cloud data to obtain clustered data representing the human being by a plurality of clusters (see Fig. 3 below, the 3D point cloud data clusters from two radars);
PNG
media_image2.png
197
276
media_image2.png
Greyscale
tracking the clusters over time to define a plurality of objects (Page 539, “The NN model estimates the position of the joints independently at each timestamp”; See Fig. 12 below, wherein the system tracks and estimates the posture over time of the subject),
PNG
media_image3.png
223
331
media_image3.png
Greyscale
wherein the tracking further comprises:
- calculating probability for every cluster in every frame that the cluster relates to the same detected object as a certain cluster in the previous frame (Page 539, section V. Temporal Correlation, “The NN model estimates the position of the joints independently at each timestamp. However, as the radar is prone to noise and the point cloud can sometimes be inaccurate, the estimate can be further refined by exploiting the temporal correlation between frames, following the assumption that the joints will not move much over one timestamp”, wherein an estimate is determined based on the confidence that a joint will be in the next frame, the confidence being determined based on the position and speed of the joint in the previous frame),
- taking into account for the clustering of body parts the velocity vectors of the points of the point cloud (Page 536, section A. Radar Characterisation, “The resulting point cloud will encode the x-y-z coordinates and velocity of the object”; Page 539, section V. Temporal Correlation, wherein when estimating if the joint will be in the next frame, a confidence estimate is determined using both the joint speed the joint position; Examiner interprets the point cloud including the velocity of the object and the confidence estimate using the joint speed to read on “taking into account … velocity vectors” for the clustering of body parts. The point cloud which represents the body/body joints uses the velocity and velocity between frames when clustering), and
e) identify body parts of the human being and positional relationships among them based on the plurality of objects, wherein the identified body parts comprise the head and/or limbs (Page 537, section B. Data Collection and Pre-Processing, “The goal of the NN is, based on the input image I, to estimate 9 heatmaps Pv∈{1...9} of size 45 × 32 for 9 joints of a person: the head, left and right shoulders, hips, elbows, and knees”; Page 538, section B. Spatial Model, “We defined five of the nine joints, the head, the left and right shoulders and hips, to be the primary joints. These joints are chosen because they have a relatively larger size and produce a stronger reflection of the mmWave signal, when compared with the elbows and the knees. Meanwhile, the positions of these joints are more important in understanding the overall posture of the person, and their relative positions regarding each other have a more regular pattern”, wherein the joints have a dependency among them and the position of the joints will contribute to the determination of the posture); and
f) analyze the positional relationships to determine the posture of the human being (Page 541, section VI. System Evaluation, “The result indicates that our system can effectively extract spatial features from the radar data and determines a person’s posture, at a competitive performance to the state-of-the-art systems in both the computer vision field and the sensor field”; see Fig. 12 above, wherein the system estimates the posture of the person standing and sitting).
While Cui discloses measuring velocity and position of the clusters / cloud data, Cui fails to explicitly disclose separating static from dynamic objects for defining the objects. Regarding the limitations of claim 7-8 and 10, Cui fails to disclose wherein the static objects are identified as patients, determining a fall risk, and sending an alarm when a fall risk surpasses a threshold.
However, Moon teaches an analogous radar sensor, wherein the radar sensor is used to manage hospital rooms (Abstract). A radar sensor is disposed on the wall or ceiling of a room to track patient movement (Page 2, last paragraph). The patient tracking unit receives data from the data processing unit and radar sensor to track information on the number of people in the hospital room, as well as the location of the patient, whether the patient goes out, information about the patient's use of the toilet, status information of the patient, including a collapsed status, position, and posture. For a collapsed status, the patient tracker 22 continuously measures the height and speed of height change of a patient. If the height change over time is greater than or equal to a predetermined value (i.e., a threshold), the position change is analyzed to determine a collapsed state. Once collapse is determined, a message is sent to the ward staff (Page 4, paragraphs 7-9). Further, the patient tracking unit can detect a person lying on a bed and tracking a patient’s movement such as when they leave (Page 2, paragraph 2 – Page 3, paragraph 1). As Moon discloses tracking multiple patients who are lying down and moving, Moon discloses the limitation of separating static and dynamic objects for defining objects. As Cui is concerned with measuring the posture of a person in, for example, an office setting, Moon teaches a radar sensing system that can track multiple people in one room being in different postures or performing different activities. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Cui to incorporate the tracking of multiple individuals as taught by Moon to track and classify more than one object in the room.
Regarding claim 3, Cui as modified further discloses wherein the local data assigned to each of the points of the point cloud data comprises a reflection magnitude (Page 537, section B. Data Collection and Pre-Processing, “We ignored the wrists and ankles, as the mmWave signal reflection from these joints is relatively weaker and less important in estimating the overall body posture”; Page 539, section B. Spatial Model, “We defined five of the nine joints, the head, the left and right shoulders and hips, to be the primary joints. These joints are chosen because they have a relatively larger size and produce a stronger reflection of the mmWave signal, when compared with the elbows and the knees … joint will contribute to the prediction of other primary joints. The other joints, the left and right elbows and knees, can have more random motions and reflect less signals”; Examiner interprets the “strong” and “weak” reflections measured from the joints to be measuring a reflection magnitude of the radar wave).
Regarding claim 4, Cui as modified further discloses wherein the local data assigned to each of the points of the point cloud data comprises a measure for a confidence of the position and/or the local data (Page 539, section V. Temporal Correlation, wherein when estimating if the joint will be in the next frame, a confidence estimate is determined using both the joint speed the joint position).
Regarding claim 5, Cui as modified discloses wherein the objects are classified to identify non-human objects and in that wherein objects representing non-human objects are excluded from the identification of the body parts of the human being and the positional relationships among them. (Page 535, Section I. Introduction, “we can filter out irrelevant information, such as clutter and noise, and locate people in the scene”; Examiner interprets the “irrelevant information, clutter and noise” to be non-human objects, as Cui discloses filtering out the data to locate people. As such, the irrelevant information, clutter, and noise is filtered out and not used in the identification of the body parts).
Regarding claim 9, Cui as modified further discloses the further steps of
mapping further points of the point cloud data, obtained from further measurements after determining the posture, to the determined posture; and
identifying points of the mapped further points lying outside of the determined posture in order to identify a posture transition (Pages 541-542, section VII. Real-Time System Integration, wherein the model is built into a real-time posture estimation system. Examiner interprets this to mean that point cloud data is continuously tracked to determine and update the posture. Cui discloses estimating walking, standing, and sitting; thus, a posture transition would necessarily be tracked when the subject being tracked moves. See Fig. 12 below of estimating the posture of a person standing and sitting).
PNG
media_image3.png
223
331
media_image3.png
Greyscale
Regarding claim 12, Cui as modified further discloses wherein a range Doppler map is obtained from the at least two dimensional radar measurements (Pages 538-539, section B. Spatial Model, “However, since the regression process of each joint is independent, the model does not consider the relative position between the joints, which sometimes leads to anatomically incorrect postures. To address this issue, and inspired by [12], we added a spatial model into our system. In [12], the authors proposed an MRF model to formulate the spatial relationship between the joints. We adapted this model into our system … We defined five of the nine joints, the head, the left and right shoulders and hips, to be the primary joints. These joints are chosen because they have a relatively larger size and produce a stronger reflection of the mmWave signal, when compared with the elbows and the knees. Meanwhile, the positions of these joints are more important in understanding the overall posture of the person, and their relative positions regarding each other have a more regular pattern”; Examiner interprets the MRF model and radar measurements disclosed by Cui to read on the range Doppler map. As disclosed by Applicant’s specification on Page 12, lines 5-8, “Range Doppler maps relate the distance of targets from a receiving antenna to their relative velocity away from or towards the receiving antenna”. The MRF model relates the joint positions relative to each other while the radar measurements measure the velocity between frames of the joints to determine a posture) and wherein information on obtained clusters, defined objects and/or identified body parts is used to identify regions of interest in the range Doppler map, the regions of interest of the range Doppler map being analyzed in order to do at least one of the following:
- identify valid radar measurements to be considered for obtaining the at least two dimensional point cloud data (Pages 539-540, section V. Temporal Correlation, wherein the NN model data is used to determine an estimate and confidence of the joints motion. If the confidence fluctuates too much or falls out of 15% of their mean values, the data will be considered “unstable”.);
Regarding claim 20, Cui discloses a system for determining a posture of a human being, comprising:
a radar system for generating measurement data from at least two dimensional radar measurements from a space accommodating the human being (Fig. 2 below, radar 1 and radar 2; Page 536, section A. Radar Characterisation, “In this research, we used the TI IWR1443 mmWave radars. As an FMCW (frequency-modulated continuous wave) radar, it sends chirp signals and detects any frequency and phase changes between the transmitted signal and reflected signal, to determine the distance and velocity of any object in front of it. The radar has three transmitting antennas and four receiving antennas … The arrangement of the transmitters enables the radar to capture both azimuth and elevation information”);
PNG
media_image1.png
183
220
media_image1.png
Greyscale
a first processor adapted to process the measurement data to obtain three dimensional point cloud data, a position and local data being assigned to each of the points of the point cloud data, the local data comprising velocity vectors of the points (Page 537, section B. Data Collection and Pre-Processing, “The radar outputs a 3D point cloud in arbitrary sizes, representing the geometric information about any subject in front of it”; Page 536, section A. Radar Characterisation, “The resulting point cloud will encode the x-y-z coordinates and velocity of the object”; Page 538, section B. Spatial Model, “The architecture of the spatial model is shown in Figure 5. The MRF operation is implemented as a convolution operation (the MRF Conv block in Figure 5), where Wcv is defined as the convolution kernel and bcv is the matrix containing the bias term. The convolution operation models how the estimate of joint c contributes to the estimate of joint v. The ReLU function is applied to the heatmaps (P) and the convolution kernels (W) before performing the convolution to ensure non-negative values and improve the stability of the network”);
a second processor adapted to cluster the point cloud data to obtain clustered data representing the human being by a plurality of clusters (see Fig. 3 below, the 3D point cloud data clusters from two radars);
PNG
media_image2.png
197
276
media_image2.png
Greyscale
a tracker adapted to track the clusters over time to define a plurality of objects (Page 539, “The NN model estimates the position of the joints independently at each timestamp”; See Fig. 12 below, wherein the system tracks and estimates the posture over time of the subject),
PNG
media_image3.png
223
331
media_image3.png
Greyscale
wherein the tracker is adapted to:
- calculate a probability for every cluster in every frame that the cluster relates to the same detected object as a certain cluster in the previous frame (Page 539, section V. Temporal Correlation, “The NN model estimates the position of the joints independently at each timestamp. However, as the radar is prone to noise and the point cloud can sometimes be inaccurate, the estimate can be further refined by exploiting the temporal correlation between frames, following the assumption that the joints will not move much over one timestamp”, wherein an estimate is determined based on the confidence that a joint will be in the next frame, the confidence being determined based on the position and speed of the joint in the previous frame),
- taking into account for the clustering of body parts the velocity vectors of the points of the point cloud (Page 536, section A. Radar Characterisation, “The resulting point cloud will encode the x-y-z coordinates and velocity of the object”; Page 539, section V. Temporal Correlation, wherein when estimating if the joint will be in the next frame, a confidence estimate is determined using both the joint speed the joint position; Examiner interprets the point cloud including the velocity of the object and the confidence estimate using the joint speed to read on “taking into account … velocity vectors” for the clustering of body parts. The point cloud which represents the body/body joints uses the velocity and velocity between frames when clustering);
e) an identifier adapted to identify body parts of the human being and positional relationships among them based on the plurality of objects, wherein the identified body parts comprise the head and/or limbs (Page 537, section B. Data Collection and Pre-Processing, “The goal of the NN is, based on the input image I, to estimate 9 heatmaps Pv∈{1...9} of size 45 × 32 for 9 joints of a person: the head, left and right shoulders, hips, elbows, and knees”; Page 538, section B. Spatial Model, “We defined five of the nine joints, the head, the left and right shoulders and hips, to be the primary joints. These joints are chosen because they have a relatively larger size and produce a stronger reflection of the mmWave signal, when compared with the elbows and the knees. Meanwhile, the positions of these joints are more important in understanding the overall posture of the person, and their relative positions regarding each other have a more regular pattern”, wherein the joints have a dependency among them and the position of the joints will contribute to the determination of the posture); and
f) an analyzer adapted to analyze the positional relationships to determine the posture of the human being (Page 541, section VI. System Evaluation, “The result indicates that our system can effectively extract spatial features from the radar data and determines a person’s posture, at a competitive performance to the state-of-the-art systems in both the computer vision field and the sensor field”; see Fig. 12 above, wherein the system estimates the posture of the person standing and sitting).
While Cui discloses measuring velocity and position of the clusters / cloud data, Cui fails to explicitly disclose separating static from dynamic objects for defining the objects.
However, Moon teaches an analogous radar sensor, wherein the radar sensor is used to manage hospital rooms (Abstract). A radar sensor is disposed on the wall or ceiling of a room to track patient movement (Page 2, last paragraph). The patient tracking unit receives data from the data processing unit and radar sensor to track information on the number of people in the hospital room, as well as the location of the patient, whether the patient goes out, information about the patient's use of the toilet, status information of the patient, including whether the collapsed, position, and posture. Further, the patient tracking unit can detect a person lying on a bed and tracking a patient’s movement such as when they leave (Page 2, paragraph 2 – Page 3, paragraph 1). As Moon discloses tracking multiple patients who are lying down and moving, Moon discloses the limitation of separating static and dynamic objects for defining objects. As Cui is concerned with measuring the posture of a person in, for example, an office setting, Moon teaches a radar sensing system that can track multiple people in one room being in different postures or performing different activities. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Cui to incorporate the tracking of multiple individuals as taught by Moon to track and classify more than one object in the room.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Cui et. al. (“Real-Time Short-Range Human Posture Estimation Using mmWave Radars and Neural Networks”), hereinafter Cui, and Moon (KR 20220071622) as applied to claim 10 above, and further in view of Collins (US 7319386)
Regarding claim 11, the combination of Cui and Moon disclose tracking multiple objects in a room and setting an alarm when a fall risk is detected, as described above. Cui as modified by Moon fails to disclose resetting the alarm condition when a caregiver approaches.
Collins teaches an analogous system for monitoring various hospital beds (Abstract), fall risks are determined and alarms are set for certain conditions of the patient (Col 8, line 66 – Col 9, line 19). The alarm may cease flashing when the system detects that a caregiver has entered the room (Col 3, lines 55-64). As Cui modified by Moon is concerned with tracking multiple patients in a hospital room, detecting collapse conditions, and sending alarms when a collapse is detected, Collins introduces a method of ceasing the alarm when a caregiver enters the room. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fall detection and alarm system of Cui and Moon to cease the alarm when a caregiver enters the room as taught by Collins, the benefit being that the caregiver can focus attention on the patient.
Claims 13-19 are rejected under 35 U.S.C. 103 as being unpatentable over Cui et. al. (“Real-Time Short-Range Human Posture Estimation Using mmWave Radars and Neural Networks”), hereinafter Cui, and Moon (KR 20220071622) as applied to claim 1 above, and further in view of Vu (US 20180049669).
Regarding claims 13, 15, and 19, Cui as modified discloses measuring the joints of shoulders and hips defining a chest region, and creating dependencies on these joints to identify other joints, as described above. Cui as modified fails to disclose identifying a chest based on breathing or heartbeat.
Regarding the limitations of claims 15 and 19, Cui as modified fails to further disclose wherein each object is classified according to its geometrical shape into a torso candidate class or a non-torso candidate class and using a frequency analysis on the single values to classify a chest or non-chest candidate.
Vu teaches an analogous system as Cui, Moon, and Vu are all in the same field of taking radar measurements. Vu teaches a breathing monitor that uses a radar navigator to track random movement of the subject, including movement of the shoulders, limbs, other body parts, and the entire body. “Using the phase-shift and signal strength information gathered by the volume estimator as inputs, the navigator detects large and small scale body movement. The navigator estimates the sleeping posture of the subject and moves the antenna accordingly to redirect the radio beam to the subject's chest upon detecting body movement” (Paragraph 0144). “Further, a one-time neural-network-based training process is designed to mine the relationship between breathing volume and chest movement for each chest area” (Paragraph 0151). To reiterate, Vu teaches detecting body movement of the chest, which is used to determine breathing volume, to redirect the radio beam to the subject’s chest. Examiner interprets this redirection as a method of identifying the chest based on the breathing movement. Additionally, Vu takes a depth-image of the subjects chest and classifies each pixel as part of the patient or as part of the background (i.e., torso or not torso), wherein high-frequency pixel fluctuations were minimized to reduce the natural fluctuations of the depth measurements (Paragraph 0191). As Cui and Moon are concerned with clustering body parts to determine a posture, Vu introduces a method of determining a chest area based on breathing movements and classifying the image as a part of the chest or not.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Cui and Moon to incorporate identifying a chest position based on breathing taught by Vu, the benefit being providing a sufficient means of identifying a desired body location.
Regarding claim 14, Cui as modified further discloses wherein a movement activity level is determined from an amplitude and/or phase of the measurement data (Page 536, section A. Radar Characterisation, wherein the frequency and phase changes between the transmitted and reflected signals are measured), wherein the activity level is classified into at least one high activity level class and into at least one low activity level class (Page 536, section A. Radar Characterisation, wherein the frequency and phase changes are used to determine distance and velocity of the object and the resulting point cloud; Page 537, section B. Data Collection and Pre-Processing, wherein the postures of standing, walking, and sitting are determined. Examiner interprets standing or sitting as a “low activity level class” and walking as a “high activity level class) and wherein the amplitude and/or phase of the measurement data relating to an activity level classified in the at least one low activity level class is analyzed to identify the position of the chest region (Page 536, section A. Radar Characterisation, wherein the frequency and phase changes are used to determine distance and velocity of the object and the resulting point cloud; Page 537, section B. Data Collection and Pre-Processing, wherein the 3D point cloud from the radar data determines the positions of the joints. Examiner interprets the joints of the left and right shoulders and the hips as defining a chest/torso region).
Regarding claim 16, Cui as modified further teaches wherein for each of the objects a single value is calculated for all the points of the point cloud assigned to the respective cluster, collected during a first sample period (See Fig. 4 below, wherein a point is assigned for each cluster in the final step, each point representing a joint),
PNG
media_image4.png
118
697
media_image4.png
Greyscale
the single value representing an activity level in the respective cluster (Fig. 12 below, wherein the joints identified above are used to estimate the posture/activity of the subject, such as standing, sitting, or walking).
PNG
media_image5.png
218
315
media_image5.png
Greyscale
Regarding claim 17, Cui as modified further discloses wherein the single value is calculated from a number of points collected and absolute velocities of each of the points (Page 536, section A. Radar Characterisation, wherein the radar data results in a point cloud, each point consisting of x-y-z coordinates and a velocity; Page 537, section B. Data Collection and Pre-Processing, wherein the 3D point cloud is cluttered and joints are determined therefrom).
Regarding claim 18, Cui as modified further discloses wherein the single values are collected over a second sample period (Page 539, “The NN model estimates the position of the joints independently at each timestamp”; See Fig. 12 below, wherein the system tracks and estimates the posture over time of the subject; Examiner interprets the system tracking postures over time as collecting the values over a multiple sample periods).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NOAH MICHAEL HEALY whose telephone number is (703)756-5534. The examiner can normally be reached Monday - Friday 8:30am - 5:30pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Sims can be reached at (571)272-7540. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NOAH M HEALY/Examiner, Art Unit 3791
/JASON M SIMS/Supervisory Patent Examiner, Art Unit 3791
Prosecution Insights