DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
The present application’s priority under Provisional US Application Number 63/508,784 is acknowledged.
Information Disclosure Statement
The information disclosure statement (IDS) submitted is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Status of Claims
Claims 1-20 are pending.
Claims 1, 7-8, and 14-15 have been amended.
Response to Amendment
Rejections Under 35 U.S.C. §101: The amendments to claims 1, 8, and 15 overcome the rejection of record. The rejection is withdrawn.
Rejections Under 35 U.S.C. §103: Claims 1, 8, and 15 have been amended to change the scope of the claimed invention. Specifically, amended claim 1 recites “the one or more sound sources including one or more traffic participants of the roadway”, and “wherein the autonomous vehicle is a roadway vehicle” which changes the scope of the claimed invention.
Response to Arguments
Rejections Under 35 U.S.C. §103: Applicant’s amendments have necessitated new grounds of rejection presented in this Office action. Accordingly, Applicant’s arguments with respect to claims 1, 8, and 15 have been considered but are moot because the arguments do not apply to the current rejection. Examiner now relies on reference Macoskey et al. (US 20220205874 A1) to serve as primary reference in light of “roadway” amendments.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 8, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Macoskey et al. (US 20220205874 A1) in view of Zimmerman et al. (US 20070025183 A1) and Nakamura et al. (US 20160059418 A1).
Regarding claim 1, Macoskey teach An autonomous vehicle (see at least FIG. 11: vehicle 1100; [0092]: “the vehicle is an at least partially autonomous vehicle”), comprising:
a microphone array (see at least [0047]: “a microphone array with n∈[0, . . . , N] microphones”) of a plurality of microphones;
a visual sensor network (see at least [0091]: “The sensor 1104 may include … LiDAR”) configured to receive visual signals;
at least one processor (see at least FIG. 3: processor 302); and
at least one memory (see at least FIG. 3: memory 314) storing instructions, which, when executed by the at least one processor, cause the at least one processor to (see at least [0110]: “The program code may be distributed using a computer readable storage medium having computer readable program instructions thereon for causing a processor to carry out aspects of one or more embodiments.”):
generate spatial beamforming maps (see at least [0004]: “an energy map via spatio-dynamic beamforming”) of a roadway (see at least [0092]: “roadway”) locating one or more sound sources (see at least [0035]: “these methods can be used to acquire information about a variety of energy sources including acoustic emitters”) using a beamforming model (see at least [0035]: “A receiver or array of receivers in conjunction with beamforming algorithms are deployed on a mobile platform that records information at various locations and uses this spatially distributed information to reconstruct a coherent model of the measured space.”) corresponding to acoustic signals received at the plurality of microphones (see at least [0045]: “sensors used for recording environmental data for spatio-dynamic beamforming such as microphones”) of the microphone array, the one or more sound sources including on or more traffic participants (see at least [0092]: “control system 1102 may segment an image (e.g., … acoustic …) or other input from sensor 1104 into one or more background classes and one or more object classes (e.g. … vehicles”) of the roadway;
generate feature maps by combining (see at least [0027]: “use the spatiotemporal information coupled with the beamformed information acquired at each location to produce a high-resolution map of the space. This map can be overlaid with visual information”) the spatial beamforming maps of the roadway with visualization maps (see at least [0027]: “visual information”; [0045]: “provide visual information … such as … location information”; [0043]: “the robotic platform may be configured to localize itself in space … to provide telemetry information. This telemetry information could include … precise coordinates in space”; [0036]: “The telemetric portion of this system may include an apparatus attached to the mobile platform that provides this information, e.g., … LiDAR”) of the roadway generated based on the visual signals received by the visual sensor network; and
control operation (see at least [0092]: “In embodiments in which the vehicle is an at least a partially autonomous vehicle, actuator 1106 may be embodied in a brake system, a propulsion system, an engine, a drivetrain, or a steering system of the vehicle. Actuator control commands may be determined such that actuator 1106 is controlled such that the vehicle avoids collisions with detected objects.”) of the autonomous vehicle to travel along trajectories generated
wherein the autonomous vehicle (see at least FIG. 11: vehicle 1100; [0092]: “the vehicle is an at least partially autonomous vehicle”) is a roadway (see at least [0092]: “roadway”) vehicle.
However, Macoskey does not explicitly teach apply a synthetic aperture expansion to the acoustic signals to increase resolution of the spatial beamforming maps;
based on the generated feature maps.
Zimmerman teach apply a synthetic aperture expansion to the acoustic signals to increase resolution (see at least [0028]: “ping to ping correlation may be performed …. The correlation may utilize … synthetic aperture processing … to … improve resolution”) of the spatial beamforming maps (see at least [0033]: “the processor 104 takes all of the incoming data. It beamforms the acoustic data and creates 3D image relative to the array face.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Zimmerman to use synthetic aperture processing. Doing so would “improve accuracy, improve resolution”, as recognized by Zimmerman in paragraph [0028].
Nakamura teaches control operation (see at least [0187]: “generates an action plan for moving toward a second direction that differs from the first direction if the acoustic signal is determined as a direct sound, based on the two-dimensional map information and the sound source direction estimated by the sound source direction determination unit; and a control unit that controls the autonomous action robot according to the action plan.”) of the autonomous vehicle to travel along trajectories generated based on the generated feature maps (see at least FIG. 16, [0183]: “FIG. 16 is a diagram for describing the example of the result of measurements”; [0184]: “In FIG. 16, reference symbol St denotes a sound source, the image of a region shown with reference symbol m201 shows two-dimensional map information”; FIG. 10 S10-S17, [0187]: “a sound source direction determination unit … that re-estimates the direction of the sound source by determining whether the acoustic signal is a reflection reflected from a reflective object or a direct sound from the sound source, based on the two-dimensional map information … and the estimated sound source direction”; [0137]: “The first sound source localization unit 20 uses each of the M acoustic signals of F frames acquired by the sound acquisition unit 10, to thereby estimate the direction angle ψfr, which is the sound source direction at the f-th frame, by means of … beam forming method”; [0021]: “a distance measurement step of performing measurements related to distance …; a map information generation step of generating two-dimensional map information …, using information of the distance”; [0058]: “The LRF sensor outputs by a wireless or wired means, to the first map information generation unit 50”; [0057]: “The first sensor 30 is a distance sensor, and is, for example, a LRF (Laser Range Finder) sensor.”), *Examiner’s interpretation: Underlined and italicized portions of quotations from Nakamura added by examiner to group acoustic related items and visual signal related items. The action plan of Nakamura is based on “sound source direction” derived from acoustic signals and “two-dimensional map information” derived from visual signals.*
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Nakamura to control operation of a vehicle using maps based on acoustic and light-based sensor information. Doing so would help to “avoid an obstacle, and move smoothly toward the sound source direction”, as recognized by Nakamura in paragraph [0025].
Regarding claim 8, Macoskey teach A computer-implemented (see at least [0110]: “The program code may be distributed using a computer readable storage medium having computer readable program instructions thereon for causing a processor to carry out aspects of one or more embodiments.”) method, comprising:
generating spatial beamforming maps (see at least [0004]: “an energy map via spatio-dynamic beamforming”) of a roadway (see at least [0092]: “roadway”) locating one or more sound sources (see at least [0035]: “these methods can be used to acquire information about a variety of energy sources including acoustic emitters”) using a beamforming model (see at least [0035]: “A receiver or array of receivers in conjunction with beamforming algorithms are deployed on a mobile platform that records information at various locations and uses this spatially distributed information to reconstruct a coherent model of the measured space.”) corresponding to acoustic signals received at a plurality of microphones (see at least [0045]: “sensors used for recording environmental data for spatio-dynamic beamforming such as microphones”) of a microphone array (see at least [0047]: “a microphone array with n∈[0, . . . , N] microphones”), the one or more sound sources including on or more traffic participants (see at least [0092]: “control system 1102 may segment an image (e.g., … acoustic …) or other input from sensor 1104 into one or more background classes and one or more object classes (e.g. … vehicles”) of the roadway;
generating feature maps by combining (see at least [0027]: “use the spatiotemporal information coupled with the beamformed information acquired at each location to produce a high-resolution map of the space. This map can be overlaid with visual information”) the spatial beamforming maps of the roadway with visualization maps (see at least [0027]: “visual information”; [0045]: “provide visual information … such as … location information”; [0043]: “the robotic platform may be configured to localize itself in space … to provide telemetry information. This telemetry information could include … precise coordinates in space”; [0036]: “The telemetric portion of this system may include an apparatus attached to the mobile platform that provides this information, e.g., … LiDAR”) of the roadway generated based on the visual signals received by the visual sensor network; and
controlling operation (see at least [0092]: “In embodiments in which the vehicle is an at least a partially autonomous vehicle, actuator 1106 may be embodied in a brake system, a propulsion system, an engine, a drivetrain, or a steering system of the vehicle. Actuator control commands may be determined such that actuator 1106 is controlled such that the vehicle avoids collisions with detected objects.”) of the autonomous vehicle to travel along trajectories generated
wherein the autonomous vehicle (see at least FIG. 11: vehicle 1100; [0092]: “the vehicle is an at least partially autonomous vehicle”) is a roadway (see at least [0092]: “roadway”) vehicle.
However, Macoskey does not explicitly teach applying a synthetic aperture expansion to the acoustic signals to increase resolution of the spatial beamforming maps;
based on the generated feature maps.
Zimmerman teach applying a synthetic aperture expansion to the acoustic signals to increase resolution (see at least [0028]: “ping to ping correlation may be performed …. The correlation may utilize … synthetic aperture processing … to … improve resolution”) of the spatial beamforming maps (see at least [0033]: “the processor 104 takes all of the incoming data. It beamforms the acoustic data and creates 3D image relative to the array face.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Zimmerman to use synthetic aperture processing. Doing so would “improve accuracy, improve resolution”, as recognized by Zimmerman in paragraph [0028].
Nakamura teaches controlling operation (see at least [0187]: “generates an action plan for moving toward a second direction that differs from the first direction if the acoustic signal is determined as a direct sound, based on the two-dimensional map information and the sound source direction estimated by the sound source direction determination unit; and a control unit that controls the autonomous action robot according to the action plan.”) of the autonomous vehicle to travel along trajectories generated based on the generated feature maps (see at least FIG. 16, [0183]: “FIG. 16 is a diagram for describing the example of the result of measurements”; [0184]: “In FIG. 16, reference symbol St denotes a sound source, the image of a region shown with reference symbol m201 shows two-dimensional map information”; FIG. 10 S10-S17, [0187]: “a sound source direction determination unit … that re-estimates the direction of the sound source by determining whether the acoustic signal is a reflection reflected from a reflective object or a direct sound from the sound source, based on the two-dimensional map information … and the estimated sound source direction”; [0137]: “The first sound source localization unit 20 uses each of the M acoustic signals of F frames acquired by the sound acquisition unit 10, to thereby estimate the direction angle ψfr, which is the sound source direction at the f-th frame, by means of … beam forming method”; [0021]: “a distance measurement step of performing measurements related to distance …; a map information generation step of generating two-dimensional map information …, using information of the distance”; [0058]: “The LRF sensor outputs by a wireless or wired means, to the first map information generation unit 50”; [0057]: “The first sensor 30 is a distance sensor, and is, for example, a LRF (Laser Range Finder) sensor.”), *Examiner’s interpretation: Underlined and italicized portions of quotations from Nakamura added by examiner to group acoustic related items and visual signal related items. The action plan of Nakamura is based on “sound source direction” derived from acoustic signals and “two-dimensional map information” derived from visual signals.*
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Nakamura to control operation of a vehicle using maps based on acoustic and light-based sensor information. Doing so would help to “avoid an obstacle, and move smoothly toward the sound source direction”, as recognized by Nakamura in paragraph [0025].
Regarding claim 15, Macoskey teach A non-transitory (see at least FIG. 3: memory 314; [0110]: “ROM”) computer-readable medium (CRM) embodying programmed instructions which, when executed by at least one processor (see at least FIG. 3: processor 302) of an autonomous vehicle (see at least FIG. 11: autonomous vehicle 1100), cause the at least one processor to perform operations comprising (see at least [0110]: “The program code may be distributed using a computer readable storage medium having computer readable program instructions thereon for causing a processor to carry out aspects of one or more embodiments.”):
generating spatial beamforming maps (see at least [0004]: “an energy map via spatio-dynamic beamforming”) of a roadway (see at least [0092]: “roadway”) locating one or more sound sources (see at least [0035]: “these methods can be used to acquire information about a variety of energy sources including acoustic emitters”) using a beamforming model (see at least [0035]: “A receiver or array of receivers in conjunction with beamforming algorithms are deployed on a mobile platform that records information at various locations and uses this spatially distributed information to reconstruct a coherent model of the measured space.”) corresponding to acoustic signals received at a plurality of microphones (see at least [0045]: “sensors used for recording environmental data for spatio-dynamic beamforming such as microphones”) of a microphone array (see at least [0047]: “a microphone array with n∈[0, . . . , N] microphones”), the one or more sound sources including on or more traffic participants (see at least [0092]: “control system 1102 may segment an image (e.g., … acoustic …) or other input from sensor 1104 into one or more background classes and one or more object classes (e.g. … vehicles”) of the roadway;
generating feature maps by combining (see at least [0027]: “use the spatiotemporal information coupled with the beamformed information acquired at each location to produce a high-resolution map of the space. This map can be overlaid with visual information”) the spatial beamforming maps of the roadway with visualization maps (see at least [0027]: “visual information”; [0045]: “provide visual information … such as … location information”; [0043]: “the robotic platform may be configured to localize itself in space … to provide telemetry information. This telemetry information could include … precise coordinates in space”; [0036]: “The telemetric portion of this system may include an apparatus attached to the mobile platform that provides this information, e.g., … LiDAR”) of the roadway generated based on the visual signals received by the visual sensor network; and
controlling operation (see at least [0092]: “In embodiments in which the vehicle is an at least a partially autonomous vehicle, actuator 1106 may be embodied in a brake system, a propulsion system, an engine, a drivetrain, or a steering system of the vehicle. Actuator control commands may be determined such that actuator 1106 is controlled such that the vehicle avoids collisions with detected objects.”) of the autonomous vehicle to travel along trajectories generated
wherein the autonomous vehicle (see at least FIG. 11: vehicle 1100; [0092]: “the vehicle is an at least partially autonomous vehicle”) is a roadway (see at least [0092]: “roadway”) vehicle.
However, Macoskey does not explicitly teach applying a synthetic aperture expansion to the acoustic signals to increase resolution of the spatial beamforming maps;
based on the generated feature maps.
Zimmerman teach applying a synthetic aperture expansion to the acoustic signals to increase resolution (see at least [0028]: “ping to ping correlation may be performed …. The correlation may utilize … synthetic aperture processing … to … improve resolution”) of the spatial beamforming maps (see at least [0033]: “the processor 104 takes all of the incoming data. It beamforms the acoustic data and creates 3D image relative to the array face.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Zimmerman to use synthetic aperture processing. Doing so would “improve accuracy, improve resolution”, as recognized by Zimmerman in paragraph [0028].
Nakamura teaches controlling operation (see at least [0187]: “generates an action plan for moving toward a second direction that differs from the first direction if the acoustic signal is determined as a direct sound, based on the two-dimensional map information and the sound source direction estimated by the sound source direction determination unit; and a control unit that controls the autonomous action robot according to the action plan.”) of the autonomous vehicle to travel along trajectories generated based on the generated feature maps (see at least FIG. 16, [0183]: “FIG. 16 is a diagram for describing the example of the result of measurements”; [0184]: “In FIG. 16, reference symbol St denotes a sound source, the image of a region shown with reference symbol m201 shows two-dimensional map information”; FIG. 10 S10-S17, [0187]: “a sound source direction determination unit … that re-estimates the direction of the sound source by determining whether the acoustic signal is a reflection reflected from a reflective object or a direct sound from the sound source, based on the two-dimensional map information … and the estimated sound source direction”; [0137]: “The first sound source localization unit 20 uses each of the M acoustic signals of F frames acquired by the sound acquisition unit 10, to thereby estimate the direction angle ψfr, which is the sound source direction at the f-th frame, by means of … beam forming method”; [0021]: “a distance measurement step of performing measurements related to distance …; a map information generation step of generating two-dimensional map information …, using information of the distance”; [0058]: “The LRF sensor outputs by a wireless or wired means, to the first map information generation unit 50”; [0057]: “The first sensor 30 is a distance sensor, and is, for example, a LRF (Laser Range Finder) sensor.”), *Examiner’s interpretation: Underlined and italicized portions of quotations from Nakamura added by examiner to group acoustic related items and visual signal related items. The action plan of Nakamura is based on “sound source direction” derived from acoustic signals and “two-dimensional map information” derived from visual signals.*
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Nakamura to control operation of a vehicle using maps based on acoustic and light-based sensor information. Doing so would help to “avoid an obstacle, and move smoothly toward the sound source direction”, as recognized by Nakamura in paragraph [0025].
Claims 2, 7, 9, 14, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Macoskey et al. (US 20220205874 A1) in view of Zimmerman et al. (US 20070025183 A1), Nakamura et al. (US 20160059418 A1), and Watanabe et al. (NPL).
Regarding claim 2, the combination of Macoskey, Zimmerman, and Nakamura and teach The autonomous vehicle of claim 1.
However, the combination of Macoskey, Zimmerman, and Nakamura does not explicitly teach wherein the synthetic aperture expansion is applied using a neural network or a convolution neural network.
Watanabe teach wherein the synthetic aperture expansion is applied using a neural network (see at least pg. 88, FIG. 3: neural network for 2-Dimensional Image Reconstruction) or a convolution neural network.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified v to incorporate the teachings of Watanabe to use a neural network for synthetic aperture expansion. Doing so would overcome the problem that acoustical images “are generally very distorted, and it is difficult to recognize the object with this information”, as recognized by Watanabe on pg 87 first paragraph under the 3. NEURAL NETWORKS header.
Regarding claim 7, the combination of Macoskey, Zimmerman, and Nakamura teach The autonomous vehicle of claim 1.
However, the combination of Macoskey, Zimmerman, and Nakamura does not explicitly teach wherein generate the feature maps further comprises provide the generated feature maps to an application including at least one of an object detection application or a future RGB frame prediction application.
Watanabe teach wherein generate the feature maps further comprises provide the generated feature maps to an application including at least one of an object detection application (see at least pg. 88, FIG. 2: “Neural Network for Object Identification”) or a future RGB frame prediction application.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Watanabe to apply image improvement for an object detection application. Doing so would overcome the problem that acoustical images “are generally very distorted, and it is difficult to recognize the object with this information”, as recognized by Watanabe on pg 87 first paragraph under the 3. NEURAL NETWORKS header.
Regarding claim 9, the combination of Macoskey, Zimmerman, and Nakamura teach The computer-implemented method of claim 8.
However, the combination of Macoskey, Zimmerman, and Nakamura does not explicitly teach wherein the applying the synthetic aperture expansion comprises applying the aperture expansion using a neural network or a convolution neural network.
Watanabe teach wherein the applying the synthetic aperture expansion comprises applying the aperture expansion using a neural network (see at least pg. 88, FIG. 3: neural network for 2-Dimensional Image Reconstruction) or a convolution neural network.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Watanabe to use a neural network for synthetic aperture expansion. Doing so would overcome the problem that acoustical images “are generally very distorted, and it is difficult to recognize the object with this information”, as recognized by Watanabe on pg 87 first paragraph under the 3. NEURAL NETWORKS header.
Regarding claim 14, the combination of Macoskey, Zimmerman, and Nakamura teach The computer-implemented method of claim 8.
However, the combination of Macoskey, Zimmerman, and Nakamura does not explicitly teach wherein the generating the feature maps further comprises providing the generated feature maps to an application including at least one of an object detection application or a future RGB frame prediction application.
Watanabe teach wherein the generating the feature maps further comprises providing the generated feature maps to an application including at least one of an object detection application (see at least pg. 88, FIG. 2: “Neural Network for Object Identification”) or a future RGB frame prediction application.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Watanabe to apply image improvement for an object detection application. Doing so would overcome the problem that acoustical images “are generally very distorted, and it is difficult to recognize the object with this information”, as recognized by Watanabe on pg 87 first paragraph under the 3. NEURAL NETWORKS header.
Regarding claim 16, the combination of Macoskey, Zimmerman, and Nakamura teach The non-transitory CRM of claim 15.
However, the combination of Macoskey, Zimmerman, and Nakamura does not explicitly teach wherein the applying the synthetic aperture expansion comprises applying the aperture expansion using a neural network or a convolution neural network.
Watanabe teach wherein the applying the synthetic aperture expansion comprises applying the aperture expansion using a neural network (see at least pg. 88, FIG. 3: neural network for 2-Dimensional Image Reconstruction) or a convolution neural network.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Watanabe to use a neural network for synthetic aperture expansion. Doing so would overcome the problem that acoustical images “are generally very distorted, and it is difficult to recognize the object with this information”, as recognized by Watanabe on pg 87 first paragraph under the 3. NEURAL NETWORKS header.
Claims 3, 10, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Macoskey et al. (US 20220205874 A1) in view of Zimmerman et al. (US 20070025183 A1), Nakamura et al. (US 20160059418 A1), and Chakrabarty et al. (NPL).
Regarding claim 3, the combination of Macoskey, Zimmerman, and Nakamura teach The autonomous vehicle of claim 1.
However, the combination of Macoskey, Zimmerman, and Nakamura does not explicitly teach wherein the synthetic aperture expansion is trained to perform neural aperture expansion on a subarray of the microphone array of the plurality of microphones.
Chakrabarty teach wherein the synthetic aperture expansion is trained (see at least first paragraph of right column on pg 16: “for the six microphones and the four microphone array, 4 and 2 CNNs are trained, respectively.”) to perform neural aperture expansion on a subarray (see at least last full paragraph of left column on pg 16: “we consider a ULA with 8 microphones with an inter-microphone distance of 2 cm. From this array, we select two sub-arrays, one with 6 microphones and the other with 4 microphones that are formed by selecting the respective number of middle microphones from the main eight element array, as shown in Fig. 4, to get a ULA with M=6 and another ULA with M=4, respectively.”) of the microphone array of the plurality of microphones.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Chakrabarty to use a subarray. Doing so would reduce computation cost, as recognized by Chakrabarty in the last paragraph of the right column on pg 8.
Regarding claim 10, the combination of Macoskey, Zimmerman, and Nakamura teach The computer-implemented method of claim 8.
However, the combination of Macoskey, Zimmerman, and Nakamura does not explicitly teach further comprising training the synthetic aperture expansion to perform neural aperture expansion on a subarray of the microphone array of the plurality of microphones.
Chakrabarty teach further comprising training (see at least first paragraph of right column on pg 16: “for the six microphones and the four microphone array, 4 and 2 CNNs are trained, respectively.”) the synthetic aperture expansion to perform neural aperture expansion on a subarray (see at least last full paragraph of left column on pg 16: “we consider a ULA with 8 microphones with an inter-microphone distance of 2 cm. From this array, we select two sub-arrays, one with 6 microphones and the other with 4 microphones that are formed by selecting the respective number of middle microphones from the main eight element array, as shown in Fig. 4, to get a ULA with M=6 and another ULA with M=4, respectively.”) of the microphone array of the plurality of microphones.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Chakrabarty to use a subarray. Doing so would reduce computation cost, as recognized by Chakrabarty in the last paragraph of the right column on pg 8.
Regarding claim 17, the combination of Macoskey, Zimmerman, and Nakamura teach The non-transitory CRM of claim 15.
However, the combination of Macoskey, Zimmerman, and Nakamura does not explicitly teach wherein the operations further comprising training the synthetic aperture expansion to perform neural aperture expansion on a subarray of the microphone array of the plurality of microphones.
Chakrabarty teach wherein the operations further comprising training (see at least first paragraph of right column on pg 16: “for the six microphones and the four microphone array, 4 and 2 CNNs are trained, respectively.”) the synthetic aperture expansion to perform neural aperture expansion on a subarray (see at least last full paragraph of left column on pg 16: “we consider a ULA with 8 microphones with an inter-microphone distance of 2 cm. From this array, we select two sub-arrays, one with 6 microphones and the other with 4 microphones that are formed by selecting the respective number of middle microphones from the main eight element array, as shown in Fig. 4, to get a ULA with M=6 and another ULA with M=4, respectively.”) of the microphone array of the plurality of microphones.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Chakrabarty to use a subarray. Doing so would reduce computation cost, as recognized by Chakrabarty in the last paragraph of the right column on pg 8.
Claims 4, 11, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Macoskey et al. (US 20220205874 A1) in view of Zimmerman et al. (US 20070025183 A1), Nakamura et al. (US 20160059418 A1), Chakrabarty et al. (NPL), and Markish et al. (US 20220214423 A1).
Regarding claim 4, the combination of Macoskey, Zimmerman, Nakamura, and Chakrabarty teach The autonomous vehicle of claim 3.
However, the combination of Macoskey, Zimmerman, Nakamura, and Chakrabarty does not explicitly teach wherein the subarray of the microphone array includes a subset of the plurality of microphones arranged in a grid pattern of 24x24.
Markish teach wherein the subarray of the microphone array includes a subset of the plurality of microphones arranged in a grid pattern of 24x24 (see at least [0432]: “non-uniform virtual MIMO antenna array 1750 may include 576 virtual elements, e.g., Nvirt = 24 * 24 = 576”; [0068]: “other aspects may be implemented with respect to, or in conjunction with, any other … acoustic signals”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Markish to use a particular grid pattern. Doing so would improve image resolution, as recognized by Markish in paragraph [0215].
Regarding claim 11, the combination of Macoskey, Zimmerman, Nakamura, and Chakrabarty teach The computer-implemented method of claim 10.
However, the combination of Macoskey, Zimmerman, and Nakamura, Chakrabarty does not explicitly teach wherein the subarray of the microphone array includes a subset of the plurality of microphones arranged in a grid pattern of 24x24.
Markish teach wherein the subarray of the microphone array includes a subset of the plurality of microphones arranged in a grid pattern of 24x24 (see at least [0432]: “non-uniform virtual MIMO antenna array 1750 may include 576 virtual elements, e.g., Nvirt = 24 * 24 = 576”; [0068]: “other aspects may be implemented with respect to, or in conjunction with, any other … acoustic signals”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Markish to use a particular grid pattern. Doing so would improve image resolution, as recognized by Markish in paragraph [0215].
Regarding claim 18, the combination of Macoskey, Zimmerman, Nakamura, and Chakrabarty teach The non-transitory CRM of claim 17.
However, the combination of Macoskey, Zimmerman, and Nakamura, Chakrabarty does not explicitly teach wherein the subarray of the microphone array includes a subset of the plurality of microphones arranged in a grid pattern of 24x24.
Markish teach wherein the subarray of the microphone array includes a subset of the plurality of microphones arranged in a grid pattern of 24x24 (see at least [0432]: “non-uniform virtual MIMO antenna array 1750 may include 576 virtual elements, e.g., Nvirt = 24 * 24 = 576”; [0068]: “other aspects may be implemented with respect to, or in conjunction with, any other … acoustic signals”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Markish to use a particular grid pattern. Doing so would improve image resolution, as recognized by Markish in paragraph [0215].
Claims 5, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Macoskey et al. (US 20220205874 A1) in view of Zimmerman et al. (US 20070025183 A1), Nakamura et al. (US 20160059418 A1), Chakrabarty et al. (NPL), and Graham et al. (US 11601749 B1).
Regarding claim 5, the combination of Macoskey, Zimmerman, Nakamura, and Chakrabarty teach The autonomous vehicle of claim 3.
However, the combination of Macoskey, Zimmerman, Nakamura, and Chakrabarty does not explicitly teach wherein the plurality of microphones of the microphone array is arranged in a grid pattern of 32x32.
Graham teach wherein the plurality of microphones of the microphone array is arranged in a grid pattern of 32x32 (see at least (22) column 5 lines 53-55: “Another embodiment of the array 116 may include 1,024 microphone elements, such as arranged in a 32x32 pattern.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Graham to use a particular grid pattern. Doing so would reduce costs, as recognized by Graham in paragraph column 1 line 59 – column 2 line 17.
Regarding claim 12, the combination of Macoskey, Zimmerman, Nakamura, and Chakrabarty teach The computer-implemented method of claim 10.
However, the combination of Macoskey, Zimmerman, and Nakamura, Chakrabarty does not explicitly teach wherein the plurality of microphones of the microphone array is arranged in a grid pattern of 32x32.
Graham teach wherein the plurality of microphones of the microphone array is arranged in a grid pattern of 32x32 (see at least (22) column 5 lines 53-55: “Another embodiment of the array 116 may include 1,024 microphone elements, such as arranged in a 32x32 pattern.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Graham to use a particular grid pattern. Doing so would reduce costs, as recognized by Graham in paragraph column 1 line 59 – column 2 line 17.
Regarding claim 19, the combination of Macoskey, Zimmerman, Nakamura, and Chakrabarty teach The non-transitory CRM of claim 17.
However, the combination of Macoskey, Zimmerman, Nakamura, and Chakrabarty does not explicitly teach wherein the plurality of microphones of the microphone array is arranged in a grid pattern of 32x32.
Graham teach wherein the plurality of microphones of the microphone array is arranged in a grid pattern of 32x32 (see at least (22) column 5 lines 53-55: “Another embodiment of the array 116 may include 1,024 microphone elements, such as arranged in a 32x32 pattern.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Graham to use a particular grid pattern. Doing so would reduce costs, as recognized by Graham in paragraph column 1 line 59 – column 2 line 17.
Claims 6, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Macoskey et al. (US 20220205874 A1) in view of Zimmerman et al. (US 20070025183 A1), Nakamura et al. (US 20160059418 A1), Lin et al. (CN 102207548 A), and Moore et al. (US 6614386 B1).
Regarding claim 6, the combination of Macoskey, Zimmerman, and Nakamura teach The autonomous vehicle of claim 1.
However, the combination of Macoskey, Zimmerman, and Nakamura does not explicitly teach wherein the instructions further cause the at least one processor to perform the synthetic aperture expansion by minimizing a mean-squared error (L2) and a spatial gradient loss (Ls) for a smaller aperture beamforming input (Id) to generate a larger aperture beamforming target output (Id').
Lin teach wherein the instructions further cause the at least one processor to perform the synthetic aperture expansion by minimizing a mean-squared error (L2) (see at least [0007]: “synthetic aperture radar imaging method using minimum mean square error estimation”) for a smaller aperture beamforming input (Id) to generate a larger aperture beamforming target output (Id').
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Lin to minimize mean square error. Doing so would “overcome[e] the defects of orthogonal ambiguity and clutter interference”, as recognized by Lin in paragraph [0007].
Moore teach wherein the applying the synthetic aperture expansion comprises performing the synthetic aperture expansion by minimizing a spatial gradient loss (see at least (63) column 14 lines 54-57: “This increased Doppler gradient enhances the spatial resolution when the system is used to create a synthetic aperture antenna though Doppler analysis of the reflected signal.”) (Ls).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Moore to minimize spatial gradient loss. Doing so would enable easy detection of a target, as recognized by Moore in column 14 line 57.
Regarding claim 13, the combination of Macoskey, Zimmerman, and Nakamura teach The computer-implemented method of claim 8.
However, the combination of Macoskey, Zimmerman, and Nakamura does not explicitly teach wherein the applying the synthetic aperture expansion comprises performing the synthetic aperture expansion by minimizing a mean-squared error (L2) and a spatial gradient loss (Ls) for a smaller aperture beamforming input (Id) to generate a larger aperture beamforming target output (Id').
Lin teach wherein the applying the synthetic aperture expansion comprises performing the synthetic aperture expansion by minimizing a mean-squared error (L2) (see at least [0007]: “synthetic aperture radar imaging method using minimum mean square error estimation”) for a smaller aperture beamforming input (Id) to generate a larger aperture beamforming target output (Id').
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Lin to minimize mean square error. Doing so would “overcome[e] the defects of orthogonal ambiguity and clutter interference”, as recognized by Lin in paragraph [0007].
Moore teach wherein the applying the synthetic aperture expansion comprises performing the synthetic aperture expansion by minimizing a spatial gradient loss (see at least (63) column 14 lines 54-57: “This increased Doppler gradient enhances the spatial resolution when the system is used to create a synthetic aperture antenna though Doppler analysis of the reflected signal.”) (Ls).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Moore to minimize spatial gradient loss. Doing so would enable easy detection of a target, as recognized by Moore in column 14 line 57.
Regarding claim 20, the combination of Macoskey, Zimmerman, and Nakamura teach The non-transitory CRM of claim 15.
However, the combination of Macoskey, Zimmerman, and Nakamura does not explicitly teach wherein the applying the synthetic aperture expansion comprises performing the synthetic aperture expansion by minimizing a mean-squared error (L2) and a spatial gradient loss (Ls) for a smaller aperture beamforming input (Id) to generate a larger aperture beamforming target output (Id').
Lin teach wherein the applying the synthetic aperture expansion comprises performing the synthetic aperture expansion by minimizing a mean-squared error (L2) (see at least [0007]: “synthetic aperture radar imaging method using minimum mean square error estimation”) for a smaller aperture beamforming input (Id) to generate a larger aperture beamforming target output (Id').
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Lin to minimize mean square error. Doing so would “overcome[e] the defects of orthogonal ambiguity and clutter interference”, as recognized by Lin in paragraph [0007].
Moore teach wherein the applying the synthetic aperture expansion comprises performing the synthetic aperture expansion by minimizing a spatial gradient loss (see at least (63) column 14 lines 54-57: “This increased Doppler gradient enhances the spatial resolution when the system is used to create a synthetic aperture antenna though Doppler analysis of the reflected signal.”) (Ls).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Macoskey to incorporate the teachings of Moore to minimize spatial gradient loss. Doing so would enable easy detection of a target, as recognized by Moore in column 14 line 57.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Prior art previously presented:
Dantrey et al. (US 20220157165 A1) teaches a system that identifies emergency vehicle locations from alarm sound analysis (see [Abstract]).
Zemp (US 20160065323 A1) teaches a system that recovers minimum mean square error of synthetic aperture data (see paragraph [0096]) for acoustic waves (see paragraph [0044]).
Guo et al. (CN 106950569 A) teaches “a multi-array element synthetic aperture focusing beam forming method based on sequential regression method. The method of the invention uses multiple array element synthetic aperture focusing beam forming and minimum mean square error criterion” (see [Abstract]).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEORGE ALCORN whose telephone number is (571) 270-3763. The examiner can normally be reached M-F, 9:30 am – 6:30 pm est.
Examiner Interview are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jelani Smith can be reached at (571) 270-3415. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/GEORGE A ALCORN III/Examiner, Art Unit 3662
/JELANI A SMITH/Supervisory Patent Examiner, Art Unit 3662