DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/22/2024 has been made record of and considered by the examiner.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference characters "105" (of FIG. 1) and "501" (of FIG. 5) have both been used to designate the “feature fusions module.” “105” (of FIG. 5) is also referred to as the detection network in [0038]. Appropriate correction is required.
The drawings are objected to because an arrow of FIG. 1 is going the wrong direction. [0029] states “Support platforms 110 may encode 112 features and transmit them to an ego platform 100.” Appropriate correction is required.
PNG
media_image1.png
483
338
media_image1.png
Greyscale
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claim 12 is objected to because of the following informalities: claim 12 recites two redundant “compress” limitations. The examiner believes this is a typo, and supported by claim 1, claim 12 should read: “extract features at each of the plurality of support platforms from the optical data with the backbone architecture, compress the features at each of the plurality of support platforms with a compression module optimized by a loss function comprising mean squared error of decompression.” Appropriate correction is required.
Claims 10-11 and 14-16 are objected to because of the following informalities: the claims recite the typo “at least one of a plurality”. Appropriate correction is required.
Claim 12 is objected to because of the following informalities: the claims recite the grammatically incorrect “plurality of support platforms an ego platform”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “agents” is used interchangeably with “platforms,” “support platforms,” “ego platform,” “ego vehicle,” “support vehicles” in claims 1-20. In claims 1 and 7, it is unclear to the examiner if the “reference platform” is an “agent”, or a separate, additional component. The examiner respectfully recommends aligning the terms amongst the claims. Appropriate correction is required.
In claim 3, “the reference platform further comprises a reference agent,” is recited, which adds further ambiguity to if the “reference platform” is an “agent”, or a separate, additional component. Appropriate correction is required.
Where applicant acts as his or her own lexicographer to specifically define a term of a claim contrary to its ordinary meaning, the written description must clearly redefine the claim term and set forth the uncommon definition so as to put one reasonably skilled in the art on notice that the applicant intended to so redefine that claim term. Process Control Corp. v. HydReclaim Corp., 190 F.3d 1350, 1357, 52 USPQ2d 1029, 1033 (Fed. Cir. 1999). The term “optical data” in claims 1, 7, and 12 is used by the claim to mean “Object detection localizes and classifies objects within image or optical data including, but not limited to, pictures, videos, LiDAR, Radar, and thermal imaging,” while the accepted meaning is “data captured using light-based sensors.”
The term “factored prioritization” in claims 4 and 18 is used by the claim to mean “Balle’s methods to compress data as Factorized Prior (FP) models [0034],” while the term “factored prioritization” never actually occurs in the specification, leading to ambiguity regarding the actual process and the scope of the claim. Appropriate correction is required.
The term “approximately” in claims 5 and 19 is a relative term which renders the claim indefinite. The term “approximately” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Additionally, it is unclear to the examiner how “average size” is bounded/determined. Additionally, it is unclear which component “its” is referring to in “of its original size.” Appropriate correction is required.
Claim 7 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being incomplete for omitting essential structural cooperative relationships of elements, such omission amounting to a gap between the necessary structural connections. See MPEP § 2172.01. The omitted structural cooperative relationships are: the claim fails to specify the component that is “receiving optical data from a plurality of agents.” Appropriate correction is required.
Claim 12 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being incomplete for omitting essential structural cooperative relationships of elements, such omission amounting to a gap between the necessary structural connections. See MPEP § 2172.01. The omitted structural cooperative relationships are: the claim fails to specify the component(s) of the ego platform “wherein the compressed features are decompressed and aligned.” Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The limitations of claims 1 and 7 under their broadest reasonable interpretation, covers mathematical concepts (taking an input, processing it, and generating an output, and the “backbone architecture” is recited with a high level of generality). This judicial exception is not integrated into a practical application because the claim does not recite a particular technical implementation of the model that provides an improvement in the technology. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the claim merely uses generic components to perform mathematical prediction and data analysis and output the predicted data. The dependent claims are rejected for similar reasons.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being clearly anticipated by Borden (‘Bandwidth constrained cooperative object detection in images’), which shares two authors with the current application. This rejection may be overcome by a declaration explaining Borden’s involvement, who is not named as an inventor in the current application.
Consider claims 1, 7, and 12, Borden discloses a method for cooperative object detection, the steps comprising:
[claim 12: one or more processors that when executing one or more instructions stored in an associated memory are configured to:]
providing a backbone architecture for feature extraction with shared weights across a plurality of agents;
capturing optical data of a scene of interest at the plurality agents;
extracting features at each of the plurality of agents from the optical data with the backbone architecture;
compressing the features at each of the plurality of agents with a compression module optimized by a loss function comprising mean squared error of decompression;
[claim 12: transmit compressed features to an ego platform, wherein the compressed features are decompressed and aligned;
an ego platform comprising:
an object recognition neural network for feature fusion;
one or more processors that when executing one or more instructions stored in an associated memory are configured to: ]
decoding compressed features from the plurality of agents at a reference platform;
fusing the features with an object recognition neural network; and
[claim 7: aligning the features with relative pose data to spatially align features collected by the plurality of agents with the reference platform, wherein spatially aligning the features is tuned by an alignment loss function; and
providing the features to an object recognition neural network];
determining a plurality of object bounding boxes and a plurality of object classes.
PNG
media_image2.png
486
663
media_image2.png
Greyscale
Dependent claims 2-6, 8-11, and 13-20 are similarly anticipated by Borden.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-9, 11-14, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (‘V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction), in further view of Choi (‘Latent-Space Scalability for Multi-Task Collaborative Intelligence’).
Consider claims 1, 7, and 12, Wang discloses a method for cooperative object detection, the steps comprising:
[claim 12: one or more processors that when executing one or more instructions stored in an associated memory (3.3; GPU) are configured to:]
providing a backbone architecture for feature extraction with shared weights across a plurality of agents (3.2 Leveraging Multiple Vehicles; “V2VNet has three main stages: (1) a convolutional network block that processes raw sensor data and creates a compressible intermediate representation, (2) a cross-vehicle aggregation stage”; 3.3 Learning);
capturing optical data of a scene of interest at the plurality agents (3.1 Which Information Should Be Transmitted, 3.2 Leveraging Multiple Vehicles; “V2VNet has three main stages: (1) a convolutional network block that processes raw sensor data and creates a compressible intermediate representation, (2) a cross-vehicle aggregation stage, which aggregates information received from multiple vehicles with the vehicle’s internal state (computed from its own sensor data)”);
extracting features at each of the plurality of agents from the optical data with the backbone architecture (3.2 LiDAR Convolution Block; “extract features from LiDAR data and transform them into bird’s-eye-view (BEV)… produce a 4x down-sampled spatial feature map. This is the intermediate representation...”);
compressing the features at each of the plurality of agents with a compression module (each vehicle compresses its intermediate representations prior to transmission) optimized by a loss function (3.2 Compression; “Balle et al.’s variational image compression algorithm to compress our intermediate representations; a convolutional network learns to compress our representations with the help of a learned hyperprior. The latent representation is then quantized and encoded losslessly with very few bits via entropy encoding. Note that our compression module is differentiable and therefore trainable”)
[claim 12: transmit compressed features to an ego platform, wherein the compressed features are decompressed and aligned (3.2, 3.2; FIG. 3, Algorithm 1; “We design our GNN to temporally warp and spatially transform the received messages to the receiver’s coordinate system.”);
an ego platform (3.2, 3.3; SDV, “itself”) comprising:
an object recognition neural network for feature fusion (3.2, 3.3; “Our loss function is cross-entropy on the vehicle classification output and smooth l1 on the bounding box parameters.”);
one or more processors that when executing one or more instructions stored in an associated memory (3.2; GPU) are configured to: ]
decoding compressed features from the plurality of agents at a reference platform (3.2 Cross-vehicle Aggregation; “After the SDV computes its intermediate representation and transmits its compressed bitstream, it decodes the representation received from other vehicles. Specifically, we apply entropy decoding to the bit stream and apply a decoder CNN to extract the decompressed feature map. We then aggregate the received information from other vehicles to produce an updated intermediate representation”);
fusing the features with an object recognition neural network (3.2 Cross-Vehicle Aggregation; “each vehicle uses a fully-connected graph neural network (GNN) as the aggregation module, where each node in the GNN is the state representation of an SDV in the scene, including itself (see Fig. 3)… warp the intermediate state of the i-th node to send a GNN message to the k-th node. We then perform joint reasoning on the spatially aligned feature maps of both nodes using a CNN… a multilayer perceptron outputs the updated intermediate representation (Algorithm 1 Line 11).”); and
[claim 7: aligning the features with relative pose data to spatially align features collected by the plurality of agents with the reference platform, wherein spatially aligning the features is tuned by an alignment loss function (3.2; Algorithm 1, FIG. 3; “After SDVs communicate messages, each receiver SDV compensates for time-delay of the received messages, and a GNN aggregates the spatial messages to compute the final intermediate representation.”; “the GNN computation is done locally by the SDV. We design our GNN to temporally warp and spatially transform the received messages to the receiver’s coordinate system.”; Imperfect Localization); and
providing the features to an object recognition neural network (3.2 Output Network; “take the feature map and exploit two network branches to output detection and motion forecasting estimates respectively. The detection output is (x, y,w, h, θ), denoting the position, size and orientation of each object.”)];
determining a plurality of object bounding boxes and a plurality of object classes (3.2 Output Network; “take the feature map and exploit two network branches to output detection and motion forecasting estimates respectively. The detection output is (x, y,w, h, θ), denoting the position, size and orientation of each object.”; 3.3; “Our loss function is cross-entropy on the vehicle classification output and smooth l1 on the bounding box parameters”).
Wang fails to explicitly disclose a compression module optimized by a loss function comprising mean squared error of decompression.
In related art, Choi discloses a compression module optimized by a loss function comprising mean squared error of decompression (Choi 3.5; loss function (Equation 2), “The first term in (4) encourages accurate reconstruction of the input image, while the third term encourages its perceptual quality. The second term is the MSE between the groundtruth feature tensor at the output of layer l=12 of YOLOv3 and the corresponding feature tensor derived from our base features Ybase.”);
providing the features to an object recognition neural network (Choi 3.1 Motivation; “At the cloud side, from the features Y”);
determining a plurality of object bounding boxes and a plurality of object classes (Choi 3.1; “sub-model f3 performs object detection, producing a collection T of bounding boxes and object classes”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the compression module optimized by a loss function comprising mean squared error of decompression of Choi into the communication method of Wang to “achieve the best compromise of having strong perception and motion forecasting performance while also satisfying existing hardware transmission bandwidth capabilities (Wang Introduction).” Choi’s system “can be trained to achieve various trade-offs between object detection accuracy and input reconstruction quality (Choi Abstract).”
Consider claim 2, Wang, as modified by Choi, discloses the claimed invention further comprising the step of: aligning the features with relative pose data to spatially align features collected by the plurality of agents with the reference platform, wherein spatially aligning the features is tuned by an alignment loss function (Wang 3; FIG. 3; Algorithm 1; Imperfect Localization).
Consider claim 3, Wang, as modified by Choi, discloses the claimed invention wherein the reference platform further comprises a reference agent for extracting reference features (Wang 3), and further comprising the step of: fusing the reference features with the compressed features (Wang 3).
Consider claims 4, 8, and 18, Wang, as modified by Choi, discloses the claimed invention wherein the compression module further comprises factorized prioritization (Wang 3.2 Compression, adaptation of Balle).
Consider claims 5 and 19, Wang, as modified by Choi, discloses the claimed invention herein the factorized prioritization reduces average size of transmission to approximately 0.2% of its original size (Wang; 3.2 Compression; FIG. 5 V2VNet + Balle .02MB vs V2VNet 10 MB ).
Consider claims 6, 9, and 20, Wang, as modified by Choi, discloses wherein the compression module further comprises autoencoder compression (Choi FIG. 2; Wang 3.2 Compression).
Consider claim 11, Wang, as modified by Choi, discloses the claimed invention wherein at least one a plurality of agents is associated with support vehicle and at least one a plurality of agents is associated with ego vehicle (Wang 3).
Consider claim 13, Wang, as modified by Choi, discloses the claimed invention the ego platform further comprising an ego optical sensor for capturing optical data (Wang 3), and
wherein one or more processors that when executing one or more instructions stored in an associated memory (Wang 3) are further configured to:
extract features from the ego optical sensor (Wang 3).
Consider claim 14, Wang, as modified by Choi, discloses the claimed invention wherein at least one a plurality of support platforms is a vehicle and the ego platform is a vehicle (Wang 3).
Consider claim 17, Wang, as modified by Choi, discloses the claimed invention wherein the ego platform is further configured to: align the features with relative pose data to spatially align features collected by the plurality of agents with the reference platform, wherein spatially aligning the features is tuned by an alignment loss function (Wang 3, FIG. 3, Algorithm 1).
Claims 10 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Wang, in further view of Choi, as applied to claims 1-9, 11-14, and 17-20 above, and further in view of Lu (‘Robust Collaborative 3D Object Detection in Presence of Pose Errors’).
Consider claim 10, Wang, as modified by Choi, fails to specifically disclose wherein at least one a plurality of agents is associated with a stationary optical sensor configured to capture a scene of interest with a surveillance system.
In related art, Lu discloses wherein at least one a plurality of agents is associated with a stationary optical sensor configured to capture a scene of interest with a surveillance system (V. Experimental Results; road-side unit).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the road-side unit of Lu into the communication methods of Wang, as modified by Choi, to include road-side information and increase accuracy (Lu V). As stated by Wang, “The crux of the problem is that the SDV and the human can only see the scene from a single viewpoint (Wang 1).”
Consider claim 16, Wang, as modified by Choi, fails to specifically disclose wherein at least one a plurality of support platforms is a stationary surveillance system.
In related art, Lu discloses wherein at least one a plurality of support platforms is a stationary surveillance system (Lu V. Experimental Results; road-side unit).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the road-side unit of Lu into the communication methods of Wang, as modified by Choi, to include road-side information and increase accuracy (Lu V). As stated by Wang, “The crux of the problem is that the SDV and the human can only see the scene from a single viewpoint (Wang 1).”
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Wang, in further view of Choi, as applied to claims 1-9, 11-14, and 17-20 above, and further in view of Shi (‘Drone Assisted Vehicular Networks: Architecture, Challenges and Opportunities’).
Consider claim 15, Wang, as modified by Choi, fails to specifically disclose wherein at least one a plurality of support platforms is a drone and the ego platform is a drone.
In related art, Shi discloses wherein at least one a plurality of support platforms is a drone and the ego platform is a drone (Shi Network components, Drone, FIG. 1).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to substitute the vehicles of Wang for the drones of Shi because “RN drones can be treated as flying vehicular nodes. They relay data for V2V communications and access infrastructures the same as vehicles (Shi Network components, Drone).”
Relevant Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Chen (‘F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds’).
Balle ("End-to-end optimized image compression.")
Assine ("Collaborative Object Detectors Adaptive to Bandwidth and Computation.")
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ASHLEY HYTREK whose telephone number is (703)756-4562. The examiner can normally be reached M-F 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Steve Koziol can be reached at (408)918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ASHLEY HYTREK/Examiner, Art Unit 2665
/Stephen R Koziol/Supervisory Patent Examiner, Art Unit 2665
Prosecution Insights