DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The claim objection has been withdrawn as the applicant has replace “also to” with “configured to” in claims 1 and 20.
The 35 USC § 112(a) rejection has been withdrawn as the new examiner believes the applicant’s arguments successfully show where the specification (page 11 line 23 through page 13 line 17) defines the steps of how the RGB data outputs the information associated with the skeletal joints.
The 35 USC § 112(b) rejection has been withdrawn as the applicant has removed the language “signal processing circuitry output signal” from claim 1.
The 35 USC § 101 rejection is being maintained as the applicant’s arguments are not found persuasive.
Regarding Step 2A Prong One, the applicant argues: “Independent claims 1 and 20 cannot be performed in a human mind because the claims are tied to circuitry, namely signal processing circuitry and data conditioning circuitry coupled to the signal processing circuitry (claim 1) and processor circuitry, signal processing circuitry, data conditioning circuitry coupled to the signal processing circuitry, and a coupled millimeter wave (mmWave) radar transceiver (claim 20). As such, the scope of these claims cannot be construed as reading on the human mind. As such, the claim does not recite matter that falls within the category of certain methods of organizing human activity, and therefore should not be treated as reciting an abstract idea. Therefore, Applicant respectfully submits claim 1 is patent eligible at Prong One of revised Step 2A”.
This is not found persuasive because the method could be performed in the mind or with pen and paper as a person could look at the point cloud data and RGB heat map (acquired in the nonsignificant extra-solution activity of the “receiving” and “generating steps”) and derive the locations of the skeletal joints and report behaviors based on the location of the skeletal joints. Essentially, the signal processing circuitry (generic computer circuitry) is acquiring and refining the data. After that, there is no reason a person could not review the data and derive conclusions about the location of skeletal joints.
Regarding Step 2A Prong Two, the applicant argues: “Applicant respectfully asserts that claim 1 integrates the recited judicial exception into a practical application of the exception at least by reporting "one or more critical patients' behaviors to appropriate personnel during periods of unsupervised care based on the at least one output signal that includes information associated with a location of each of a plurality of skeletal joints for each of at least a portion of the one or more objects" (emphasis added). The above quoted additional element(s) use the judicial exception in a meaningful way beyond generally linking the use of the judicial exception to a particular technological environment by providing a system to report critical patients' behaviors to provide healthcare during periods of unsupervised care”.
This is not found persuasive because simply reporting the patient behavior does not qualify as integrating the recited judicial exception into a practical application. The step of reporting is another step that could be performed by a person. Within sensing, the typical means to achieve integrating the recited judicial exception into a practical application is to either interact with the sensing devices in a way that makes them no longer generic sensors or adding a treatment step as a result of the sensing. However, reporting the sensed data to appropriate personnel does not qualify.
The 35 USC § 103 rejection is being modified to address the applicant’s new claim language. The new limitation is addressed by Lane. The following has been added to the previous rejection:
“Further, Nakayama in view of Yamagata and Ball teach generating, using the data representative of point cloud intensity information, at least one output signal that includes information associated with a location of each of a plurality of skeletal joints for each of at least a portion of the one or more objects (see the rejection above for Nakayama) but does not teach reporting one or more critical patients' behaviors to appropriate personnel during periods of unsupervised care based on the at least one output signal that includes information associated with a location of each of a plurality of skeletal joints for each of at least a portion of the one or more objects.
Lane in the same field of endeavor: patient monitoring system, teaches reporting one or more critical patients' behaviors to appropriate personnel during periods of unsupervised care based on the at least one output signal that includes information associated with a location of each of a plurality of skeletal joints for each of at least a portion of the one or more objects (figures 6 and 7, elements 310, 312, 330, 332 [0073], [0074], [0076]. Note: Subject position 332 will read on reporting a location of a plurality of skeletal joints. [0076] states “The subject position 332 relates to a physical condition of the subject, such as arrangement, location, movement, posture, pose, or physical characteristics of the subject”. The notification step 312 will read on reporting the patients’ behaviors).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention, to have modified the system of Nakayama in view of Yamagata and Ball with reporting one or more critical patients' behaviors to appropriate personnel during periods of unsupervised care based on the at least one output signal that includes information associated with a location of each of a plurality of skeletal joints for each of at least a portion of the one or more objects of Lane in order to generate a notification to a healthcare practitioner so that the healthcare practitioner assists the patients (e.g. see [0021] of Lane).”
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step One
The claim(s) recite(s) a system and method to detect the position of a plurality of skeletal joints.
Step 2A Prong One
Regarding claims 1 and 20, the limitations of “generate or determine, using the data representative of point cloud intensity information, at least one output signal that includes information associated with a location of each of a plurality of skeletal joints for each of at least a portion of the one or more objects” and “report one or more critical patients' behaviors to appropriate personnel during periods of unsupervised care based on the at least one output signal that includes information associated with a location of each of a plurality of skeletal joints for each of at least a portion of the one or more objects” are processes, as drafted that covers performance of limitations that can be performed by a human by reading images under the broadest reasonable interpretation standard. For example, determining a location of skeletal joints using point cloud density information encompasses nothing more than a user reviewing the point cloud density data on the object in an image of a patient. If a claim, under its broadest reasonable interpretation, covers performance of the limitation in human mind, then it falls within the “Mental Processes” grouping of abstract ideas. See MPEP 2106.04(a)(2)(III).
Step 2A Prong two
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “receiving at least one millimeter wave (mmWave) radar input signal that includes information associated with one or more object,” “generate a point cloud output signal containing multi-dimensional data associated with the one or more objects,“ and “generate and determine a data conditioning output signal that includes data representative of point cloud intensity information using at least a portion of the multi-dimensional data the received.” Receiving data and generating signals generated by processing circuitry is nothing more than mere pre-solution activity of data gathering. Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of using receiving and generating data amounts to no more than mere pre-solution activity of data gathering.
Further, the limitation of “wherein the intensity information is a heat map where the data conditioning circuitry also to assign a Red-Green- Blue (RGB) pixel value corresponding to a mmWave reflectance of each respective point in the point cloud” further limits the type of point cloud intensity information and amounts to no more than mere pre-solution activity of data gathering. Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Claims 2-4 recite the additional elements of “one or more mmWave radar transceivers to generate the at least one mmWave radar input signal that includes the data associated with each of the one or more objects detected within a respective field-of-view of each of the one or more mmWave transceivers;
wherein the one or more mmWave radar transceivers comprise a first mono-planar mmWave transceiver aligned along a first detection plane and a second mono- planar mmWave transceiver aligned along a second detection plane orthogonal to the first detection plane, the first mono-planar mmWave transceiver and the second mono-planar mmWave transceiver to generate the data associated with each of the one or more objects detected within the field-of- view of the first mono-planar mmWave transceiver and the second mono-planar mmWave transceiver; and
wherein the one or more mmWave radar transceivers comprise at least one multi-planar mmWave transceiver, the at least one multi-planar mmWave transceiver to provide the data associated with each of the one or more objects detected within the field-of-view of the at least one multi-planar mmWave transceiver” which further limits the data generated by the mmWave transceiver data and amounts to no more than mere pre-solution activity of data gathering as set forth above for claims 1.
Claim 5 recites the additional element of “output device to display the at least one output signal that includes information associated with the location of each of the plurality of skeletal joints for each of at least a portion of the one or more objects included in the at least one mmWave radar input signal” which is nothing more than mere post-solution activity of displaying as set forth above for claim 1.
Claim 6 recites the limitation of “the (Al) circuitry comprises a the convolutional neural network further comprising circuitry to detect a pose of each of the one or more objects included in the at least one mmWave radar input signal using the location of each of the plurality of skeletal joints for each respective one of the one or more objects included in the at least one mmWave radar input signal” which further limits the AI circuitry used to detect the data and amounts to no more than mere pre-solution activity of data gathering as set forth above for claim 1.
Claim 7 recites the limitations of “the point cloud output signal generated by the signal processing circuitry comprises at least one of: object clustering data or tracking data” which further limits the details of the point cloud data which is nothing more than mere pre-solution activity of data gathering as set forth above for claim 1.
Claim 8 recites the limitations of “the multi-dimensional data includes, for each point on the one or more objects included in the at least one mmWave radar input signal;
radial velocity data;
angle data; range data; and
reflection strength” which further limits the multi-dimensional point cloud data which is nothing more than mere pre-solution activity of data gathering as set forth above for claim 1.
Claim 9 recites the limitations of “the data conditioning output signal comprises a plurality of output signals including:
a first data conditioning output signal that includes depth-azimuth (XY) and reflection intensity data in the form of an NxNx3 image; and
a second data conditioning output signal that includes depth-elevation (XZ) and reflection intensity data” which further limits the data conditioning output signal which is nothing more than mere pre-solution activity of data gathering as set forth above for claim 1.
Claim 10 recites the limitations of “the AI circuitry comprises convolutional neural network (CNN) circuitry, the CNN circuitry comprises:
first neural network circuitry to receive the first data conditioning output signal that includes depth-azimuth (XY) and reflection intensity data to provide a first NxNx128 output signal;
second neural network circuitry to receive the a second data conditioning output signal that includes depth-elevation (XZ) and reflection intensity data to provide a second NxNx128 output signal;
data concatenation circuitry to concatenate the first NxNx128 output signal with the second NxNx128 output signal to generate an NxNx256 output tensor; flattening circuitry to flatten the NxNx256 output tensor; and multilayer perceptron circuitry to generate the at least one output signal that includes information associated with the location of each of the plurality of skeletal joints for each of at least a portion of the one or more objects” which further limits the CNN of the AI circuitry which is nothing more than mere pre-solution activity of data gathering as set forth above for claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Nakayama (US 2018/0192919 A1) in view of Yamagata et al. (US 2020/0289006 A1, hereinafter "Yamagata"), Ball (US 2019/0265714 A1) and Lane (US 2019/0053707 A1). Nakayama in view of Yamagata, Ball and Lane hereinafter referred to as “modified Nakayama.”
Regarding claims 1 and 20, Nakayama teaches a system to detect the position of a plurality of skeletal joints (pars. [0064]: This frequency range makes it possible to extract living body components affected by vital activities at a part of living body 50 by the movement of the heart, lungs, diaphragm or other internal organs, or vital activities of the hands and the legs”; examiner considers that by teaching a system tracking vital activities of the hands and legs, the system will detect a position of a plurality of skeletal joints, the wrist, ankles, elbow etc., would comprises joints the system is tracking the position of. Further, [0066] teach about tracking/detecting positions of body parts), the system comprising:
signal processing circuitry (par. [0031]: a sensor with antennae may transmit and receive signals) to:
receive at least wave radar input signal that includes information associated with one or more objects (par. [0057]: antennae may receive a wave signal); and
generate a point cloud output signal containing multi-dimensional data associated with the one or more objects (pars. [0004]: estimates a three-dimensional position, [0057]: receiving antennae may receive wave signals and record coordinates of object locations);
data conditioning circuitry coupled to the signal processing circuitry (par. [0074]: RCS score calculator), the data conditioning circuitry to:
receive the point cloud output signal generated by the signal processing circuitry (par. [0074]: using the living body components and the estimated position”; RCS calculator takes the output of a plurality of object locations received by the antennae); and
generate a data conditioning output signal that includes data representative of point cloud intensity information using at least a portion of the multi-dimensional data the received signal processing circuitry output signal (par. [0074]: The RCS calculator 440 calculates a propagation distance from the calculated distance RT and distance RR and calculates the RCS using the calculated propagation distance and the intensity of the living body components).
Nakayama teaches further circuitry (par. [0075] motion estimator) to:
receive the data conditioning output signal (par. [0075]); and
generate, using the data representative of point cloud intensity information, at least one output signal that includes information associated with a location of each of a plurality of skeletal joints for each of at least a portion of the one or more objects (par. [0075]: The motion estimator 450 estimates the motion of the living body 50 indicating temporal changes of the three-dimensional position estimated by the position estimation processor 430 and the RCS value calculated by the RCS calculator 440, [0082]: information indicating the correspondence between a motion of the living body 50 and multiple model codes indicating the temporal changes of the RCS value and the vertical position which is a position of the living body 50 in the vertical direction relative to the sensor 10. The motions of the living body 50 associated in the correspondence information 42 include falling down, sitting on a chair, sitting on a floor, standing up from a chair, standing up from a floor, jumping, and turning direction, as illustrated in FIG. 9-[0082] the motion estimator will take the output propagation and intensity results from the RCS score calculator and use this data to output data associated with locations of a plurality of skeletal joints. The result of this can be a plurality of states (falling down, sitting, jumping etc.) seen in Fig 9, where multiple locations of skeletal joints would be known).
Nakayama does not teach wherein the intensity information is a heat map where the data conditioning circuitry also to assign a Red-Green- Blue (RGB) pixel value corresponding to a mmWave reflectance of each respective point in the point cloud; and
wherein artificial intelligence (Al) is used as part of the further circuitry, but does note prior art where machine learning is used to estimate the state of a living body (par. [0002]).
However, Yamagata, in the same field of endeavor: a processing device for obtaining an intensity distribution of a biomedical signal discloses wherein the intensity information is a heat map where the data conditioning circuitry also to assign a Red-Green- Blue (RGB) pixel value corresponding to a mmWave reflectance of each respective point in the point cloud (Figs. 11, 14 and 15 (611) and pars. [0151], [0152]: …biomedical signals computed and obtained by the analyzer 206, each of which indicates the signal strength at a position inside the brain…intensity distribution) of the signal strength of the biomedical signals, which is specified by the time and frequency, is expressed by color…on the heat map 611, colors are indicated on a pixel-by-pixel basis by pixel values indicative of colors (for example, red, green, and blue (RGB)) that indicate the differences in signal strength) for the purpose of marking and displaying critical sites of the target area of the subject (par. [0145]).
Therefore, it would be obvious to one of ordinary skill in the art at the time of invention to one of ordinary skill in the art at the time of invention to have modified the system of Nakayama to include RGB color indications of the signal strength as taught by Yamagata in order to mark and display critical sites of the subject.
The Nakayama and Yamagata combination does not disclose wherein artificial intelligence (Al) is used as part of the further circuitry, but does note prior art where machine learning is used to estimate the state of a living body (Nakayama, par. [0002]).
However, Ball in the same field of endeavor: sensors and sensor processing that fuses sensor data to locate reflections, teaches a method of identifying objects based on point cloud information input from a sensor wherein artificial intelligence (Al) is used to determine the state of an object (pars. [0075], [0187]: … convolutional neural networks (R-CNN)…first runs the entire input image through some convolutional layers to obtain a feature map) to provide the benefit of simplicity to compute while outperforming other linear and non-linear regression methods (par. [0229]).
Therefore, it would be obvious to one of ordinary skill in the art at the time of invention to one of ordinary skill in the art at the time of invention to have modified the system of Nakayama and Yamagata with the AI processing of point cloud information as taught by Ball because this modification is the use of known technique (a neural network AI that provides output on the state of an object) to improve similar devices (radar/lidar object tracking systems) in the same way (AI processing systems may provide a system with accurate or fast processing and output of the object states, or allow for a specially trained system to recognize specific object states).
Nakayama in view of Yamagata and Ball teach a system wherein microwaves may be used to send/receive waves for a user’s body (see Nakayama [0054]) but it does not teach wherein the waves are mmWave radar waves.
Lane in the same field of endeavor: patient monitoring system, teaches a patient monitoring system (Title, par. [0020]: system detects a current status of a patient, including ambulation) wherein circuitry receives at least one millimeter wave (mmWave) radar input signal that includes information associated with one or more objects (pars. [0039]-[0042] mmWave radar may be used to detect locations of a user) to provide the benefit of a sensor that can measure range, velocity, and angle between the sensors and objects in high accuracy with waves that can penetrate materials, such as plastic, drywall and clothing, and are highly directions and distinguish two nearby objects (par. [0042]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention, to have modified the system of Nakayama in view of Yamagata and Ball with the mmWave radar of Lane in order to provide the benefit of a sensor that can measure range, velocity, and angle between the sensors and objects in high accuracy with waves that can penetrate materials, such as plastic, drywall and clothing, and are highly directions and distinguish two nearby objects.
Further, Nakayama in view of Yamagata and Ball teach generating, using the data representative of point cloud intensity information, at least one output signal that includes information associated with a location of each of a plurality of skeletal joints for each of at least a portion of the one or more objects (see the rejection above for Nakayama) but does not teach reporting one or more critical patients' behaviors to appropriate personnel during periods of unsupervised care based on the at least one output signal that includes information associated with a location of each of a plurality of skeletal joints for each of at least a portion of the one or more objects.
Lane in the same field of endeavor: patient monitoring system, teaches reporting one or more critical patients' behaviors to appropriate personnel during periods of unsupervised care based on the at least one output signal that includes information associated with a location of each of a plurality of skeletal joints for each of at least a portion of the one or more objects (figures 6 and 7, elements 310, 312, 330, 332 [0073], [0074], [0076]. Note: Subject position 332 will read on reporting a location of a plurality of skeletal joints. [0076] states “The subject position 332 relates to a physical condition of the subject, such as arrangement, location, movement, posture, pose, or physical characteristics of the subject”. The notification step 312 will read on reporting the patients’ behaviors).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention, to have modified the system of Nakayama in view of Yamagata and Ball with reporting one or more critical patients' behaviors to appropriate personnel during periods of unsupervised care based on the at least one output signal that includes information associated with a location of each of a plurality of skeletal joints for each of at least a portion of the one or more objects of Lane in order to generate a notification to a healthcare practitioner so that the healthcare practitioner assists the patients (e.g. see [0021] of Lane).
Regarding claim 2, the Nakayama, Yamagata and Ball combination teaches systems substantially as claimed in claim 1, further comprising:
one or more radar transceivers to generate the at least one radar input signal that includes the data associated with each of the one or more objects detected within a respective field-of-view of each of the one or more transceivers (Nakayama, Fig. 1 and par. [0054] receiving antennae 30 may receive microwave signals associated with object that are present in its range/target areas, Fig 1 and 3).
The combination teaches a system wherein microwaves may be used to send/receive waves for a user’s body (Nakayama, [0054]) but it does not teach wherein the waves are mmWave radar waves.
Lane teaches a patient monitoring system (Title and par. [0020]: system detects a current status of a patient, including ambulation) wherein circuitry receives at least one millimeter wave (mmWave) radar input signal that includes information associated with one or more objects (pars. [0039]-[0040]: detecting different objects and/or different portions of a single object, which can be used to map different objects and/or different portions of an object within an area that is being monitored by the sensing module 120, [0041]: mmWave radar antennae may be used to detect locations of a user) to provide the benefit of waves that can measure range, velocity, and angle between the sensors and objects in high accuracy, can penetrate materials, such as plastic, drywall and clothing, and are highly direction, and have large absolute bandwidths and thus can be used to distinguish two nearby objects (par. [0042]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention, to have modified the system of Nakayama in view of Yamagata and Ball with the mmWave radar of Lane because millimeter waves can be used for sensing, imaging, and communications. Millimeter sensors can measure range, velocity, and angle between the sensors and objects in high accuracy. Millimeter waves can penetrate materials, such as plastic, drywall and clothing, and are highly directions. Millimeter waves have large absolute bandwidths and thus can be used to distinguish two nearby objects.
Regarding claim 3, modified Nakayama teaches systems substantially as claimed in claim 2, wherein the one or more radar transceivers comprise a first mono-planar transceiver aligned along a first detection plane and a second mono-planar transceiver aligned along a second detection plane orthogonal to the first detection plane (Nakayama, Fig. 3 and par. [0054]: Here, φ.sub.T is an angle formed between a first reference direction which is a direction on the horizontal plane and arbitrarily set relative to the transmitting antenna 20, and a first living body direction which is a direction from the transmitting antenna 20 toward the living body 50 [0054], one of ordinary skill in the art would recognize this arbitrary angle may be any value that relates angle between the transceivers 20, 30, and through a process of routine optimization could arrive at 90 degrees for , φ.sub.T, thus placing the two wave directions orthogonal from one another; Fig 3 shows the transceivers arrayed at different angles), the first mono-planar transceiver and the second mono-planar transceiver to generate the data associated with each of the one or more objects detected within the field-of-view of the first mono-planar transceiver and the second mono-planar transceiver (par. [0054]: When (x.sub.b, y.sub.b, z.sub.b) is the center coordinates of a part at which the living body 50 performs vital activities, the directions (θ.sub.T, θ.sub.R, φ.sub.T, φ.sub.R) and the coordinates (x.sub.b, y.sub.b, z.sub.b) can be mutually converted according to the positional relationship between the transmitting antenna 20, the receiving antenna 30, and the living body 50).
Nakayama in view of Yamagata and Ball teaches a system wherein microwaves may be used to send/receive waves for a user’s body (Nakayama, par. [0054]) but it does not teach wherein the waves are mmWave radar waves but Lane rectifies this missing element in the same manner as seen above in claim 1.
Regarding claim 4, modified Nakayama teaches systems substantially as claimed in claim 2, wherein the one or more radar transceivers comprise at least one multi-planar transceiver, the at least one multi-planar transceiver to provide the data associated with each of the one or more objects detected within the field-of-view of the at least one multi-planar transceiver (Nakayama, par. [0055]: transmitting antenna may have NxN elements with multiple planar/coordinate (x,y,z) relationships).
Nakayama in view of Yamagata and Ball teach a system wherein microwaves may be used to send/receive waves for a user’s body (Nakayama, par. [0054]) but it does not teach wherein the waves are mmWave radar waves but Lane rectifies this missing element in the same manner as seen above in claim 1.
Regarding claim 5, the Nakayama, Yamagata and Ball combination, teaches systems substantially as claimed in claim 1, except wherein there is at least one output device to display the at least one output signal that includes information associated with the location of each of the plurality of skeletal joints for each of at least a portion of the one or more objects included in the at least one radar input signal.
Lane teaches at least one output device to display the at least one output signal that includes information associated with the location of each of the plurality of skeletal joints for each of at least a portion of the one or more objects included in the at least one radar input signal (Fig. 7 and pars. [0074]-[0075] a display may present information to a user on a the measure state/position of a patient) to provide the benefit of determining the desired position of a subject or predict what the subject is doing or wants to do (par. [0076]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention, to have modified the system of Nakayama to include a display to present information on the location of a plurality of locations of a patient to provide the benefit of determining the desired position of a subject or predict what the subject is doing or wants to do.
Further, Nakayama in view of Ball teach a system wherein microwaves may be used to send/receive waves for a user’s body (see Nakayama [0054]) but it does not teach wherein the waves are mmWave radar waves but Lane rectifies this missing element in the same manner as seen above in claim 1.
Regarding claim 6, modified Nakayama teaches systems substantially as claimed in claim 1, wherein there is circuitry to detect a pose of each of the one or more objects included in the at least one radar input signal using the location of each of the plurality of skeletal joints for each respective one of the one or more objects included in the at least one radar input signal (Nakayama, pars. [0075]-[0082] the motion estimator will take the output propagation and intensity results from the RCS score calculator and use this data to output data associated with locations of a plurality of skeletal joints. The result of this can be a plurality of states (falling down, sitting, jumping etc.) seen in Fig 9, where multiple locations of skeletal joints would be known)).
Nakayama in view of Yamagata and Lane does not teach wherein the (AI) artificial intelligence comprises a or a convolutional neural network is used as part of this further circuitry, but does note prior art where machine learning is used to estimate the state of a living body (par. [0002]).
Ball teaches a method of identifying objects based on point cloud information input from a sensor wherein a convolutional neural network is used to determine the state of an object (pars. [0075][0187][0229]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention, to have modified the system of Nakayama, Yamagata and Lane with the AI processing of point could information as taught by Ball because this modification is the use of known technique (a neural network AI that provides output on the state of an object) to improve similar devices (radar/lidar object tracking systems) in the same way (AI processing systems may provide a system with accurate or fast processing and output of the object states, or allow for a specially trained system to recognize specific object states).
Further, Nakayama in view of Yamagata and Ball teach a system wherein microwaves may be used to send/receive waves for a user’s body (see Nakayama [0054]) but it does not teach wherein the waves are mmWave radar waves but Lane rectifies this missing element in the same manner as seen above in claim 1.
Regarding claim 7, modified Nakayama teaches systems substantially as claimed in claim 1, wherein the point cloud output signal generated by the signal processing circuitry comprises at least one of: object clustering data or tracking data (Nakayama, pars. [0075]-[0082] a plurality of points, corresponding to parts/areas of a user’s body are tracked for motion over time; examiner considers that when preforming this action the system of Ball references a coordinate system for a plurality of points and retains tracking data for this cloud of points).
Regarding claim 8, modified Nakayama teaches systems substantially as claimed in claim 1, where the system generates for the point cloud output signal containing the multi-dimensional data corresponding to the one or more objects (Nakayama, pars. [0054]-[0057] receiving antennae may receive wave signals and record coordinates of object locations; a plurality of points corresponding to a plurality of locations of a user may be recorded, comprising a point cloud) further comprises:
generating, by the signal processing circuitry, a four-dimensional point cloud output signal that includes, for each point in each of the one or more objects, data representative of:
a radial velocity of the respective point included in the detected object;
an angle of the respective point included in the detected object;
a range to the respective point included in the detected object;
a reflection strength of the respective point included in the detected object (Nakayama, pars. [0066] the positions of a plurality of points is calculated and tracked, this includes data representing the recoded angles of waves received by the antennae which can pe translated into position/range/radial velocity of the object; further, [0074] an motion/velocity and intensity data may be included in the point information as well).
Regarding claim 9, modified Nakayama teaches systems substantially as claimed in claim 1, wherein the data conditioning output signal comprises a plurality of output signals including:
a first data conditioning output signal that includes depth-azimuth (XY) and reflection intensity data in the form of an NxNx3 image; and
a second data conditioning output signal that includes depth-elevation (XZ) and reflection intensity data (Nakayama, pars. [0054]-[0055]: directions (θ.sub.T, θ.sub.R, φ.sub.T, φ.sub.R and the coordinates (x.sub.b, y.sub.b, z.sub.b) can be mutually converted ,[0054]: these data points may further include intensity information at least three transmitting antenna elements 21 of the N transmitting antenna elements 21 are arranged at positions different in the vertical direction and the horizontal direction; both two component signals may be collected, one for R and one for T). Examiner considers the teaching of Nakayama where data is stored in an NxM matrix based on the positions (Nakayama, par. [0063]), based on this disclosure, the storage of the data output in an NxNx3 image would be obvious to one of ordinary skill in the art searching to store data with additional dimensions to record, i.e. to store intensity information with the positions, additional dimensions may be added to the data storage matrix).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Nakayama (US 2018/0192919 A1) in view of Yamagata et al. (US 2020/0289006 A1, hereinafter "Yamagata"), Ball (US 2019/0265714 A1) and Lane (US 2019/0053707 A1), as applied to claim 9 above, and further in view of Vajda (US 20190172224 A1). Nakayama in view of Yamagata, Ball and Lane hereinafter referred to as “modified Nakayama.”
Regarding claim 10, modified Nakayama teach systems substantially as claimed in claim 9, wherein the circuitry to receive the first data conditioning output signal that includes depth-azimuth (XY) and reflection intensity data to provide a first NxNx128 output signal (Nakayama, pars. [0054]-[0055]) Nakayama teaches wherein both two component signals may be collected, one for R and one for T (θ.sub.T, θ.sub.R, φ.sub.T, φ.sub.R) these data points may further include intensity information [0074], based on the disclosure this data may be stored in a matrix of appropriate size, the use of a NxNx128 sized output signal would be obvious to store/transmit data form a plurality of time points);
second neural network circuitry to receive the a second data conditioning output signal that includes depth-elevation (XZ) and reflection intensity data to provide a second NxNx128 output signal (Nakayama, pars. [0054]-[0055] Nakayama teaches wherein both two component signals may be collected, one for R and one for T (θ.sub.T, θ.sub.R, φ.sub.T, φ.sub.R); and
multilayer perceptron circuitry to generate the at least one output signal that includes information associated with the location of each of the plurality of skeletal joints for each of at least a portion of the one or more objects (pars. [0075]-[0082] the motion estimator will take the output propagation and intensity results from the RCS score calculator and use this data to output data associated with locations of a plurality of skeletal joints. The result of this can be a plurality of states (falling down, sitting, jumping etc.) seen in Fig 9, where multiple locations of skeletal joints would be known).
Modified Nakayama does not teach data a convolution neural network/artificial intelligence comprising:
concatenation circuitry to concatenate the first NxNx128 output signal with the second NxNx128 output signal to generate an NxNx256 output tensor;
flattening circuitry to flatten the NxNx256 output tensor.
However, Vajda, in the same field of endeavor: mapping an image of a body, teaches “a machine-learning model may output a bounding box 110 that surrounds a detected instance of an object type, such as a person” comprising a convolution neural network/artificial intelligence (par. [0035] system may use a convolutional neural network) comprising:
concatenation circuitry to concatenate (par. [0039] “The results of each may then undergo filter concatenation to generate the output of the inception module.”) the first NxNx128 output signal with the second NxNx128 output signal to generate an NxNx256 output tensor ([0065]-[0069] a tensor may be generated from the two input signals by combining/concatenating them in a layer of the CNN, this may result in an output that has double the size of the original inputs (i.e. it increases from size from 128 to 256));
flattening circuitry to flatten the NxNx256 output tensor and multilayer perceptron circuitry to generate the at least one output signal that includes information associated with the location of a body (par. [0070] Fig 6, at the end of the method steps the output may comprises a bounded box surrounding an identified area of the image, this corresponds with the ability of Nakayama to identify the joint/poses of a body position and thus this CNN technique may replace that of Nakayama to use artificial intelligence/a CNN to determine location of joints) to provide the benefit of automatic detection and processing objects appearing in images for body identification that is operable on lower end processing systems (par. [0006]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention, to have modified the system of modified Nakayama with the CNN with concatenation layers and tensor calculations for determining body positions as taught in Vajda in order to provide the benefit of automatic detection and processing objects appearing in images for body identification that is operable on lower end processing systems.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHILIP C EDWARDS whose telephone number is (571)270-1804. The examiner can normally be reached Mon-Fri, 9:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Unsu Jung can be reached on 571-272-8506. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/P.C.E/Examiner, Art Unit 3792
/AMANDA L STEINBERG/Examiner, Art Unit 3792
Prosecution Insights