Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims of U.S. Patent No. 11748901 and U.S. Patent No. 12223670. Although the claims at issue are not identical, they are not patentably distinct from each other because they are obvious variations of each other. Below are claims correspondence relationship:
Instant Application
1
2
3
4
5
6
7
8
9
10
12223670
1 +6
2
3
4
5
7
1
8 +13
9
10
11748901
1
2
3
4
5
7
1
8
9
10
Instant Application
11
12
13
14
15
16
17
18
19
20
12223670
11
12
14
8
15+19
16
17
18
20
15
11748901
12
8
14
8
15
16
17
18
20
15
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-6, 8-13, 15-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over State et al. (US 2021/0201571 A1).
Regarding claim 1, State teaches:
A computer system for generating a three-dimensional point cloud from two- dimensional data comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the computer system ([0065], “The system may comprise a processor coupled to a memory and a graphical user interface (GUI), the memory having recorded thereon a computer program comprising instructions for performing the method.”) to:
receive, via one or more processors, a plurality of two-dimensional images; ([0037], “The 20 images may comprise images captured by one or more physical cameras each having one or more respective sensors, such as photographs. The providing of the 2D images may comprise such capturing for at least part of the 2D images, and/or receiving from a distant system and/or retrieving in a memory at least (another) part of the 2D images.” [0042], “In examples, the method may comprise a user-machine interaction wherein a user captures the 2D images with a video camera by moving the video camera around a fixed real object, for example at a free distance from the real object. The video camera may be standard and capture video frames according to a perspective projection.”) and
analyze, via one or more processors, each of the plurality of two-dimensional images using a trained deep artificial neural network to generate a respective set of labeled points, ([0052], “A contour probability map is, as known, a map which has pixel values each representing a probability of contour presence. Such a map may be computed in any know manner, for example based on a machine-learning scheme implemented on an annotated dataset of 2D images associated with contour probability maps or contour maps (a contour map being a map whose pixel values each represent binarily contour presence or absence). Such a machine-learning may for example be performed according to or similarly to the teaching of paper by S. Xie, Z. Tu., Holistically-nested edge detection, in Proceedings of the IEEE international conference on computer vision (pp. 1395-1403), 2015, which is incorporated herein by reference. The edge detection method described in this paper is based on the learning of a network for predicting the edges of an image, by supervised training on an image dataset where annotated the edges. The network extracts multiscale predictions, and the classification loss for each pixel compares the cross-entropy between the binary mask ground-truth of the edges and the prediction of the edges at each scale. The predictions at each scale are then aggregated to give a final prediction.”) each of the labeled points corresponding to a respective class label describing an object depicted in the two-dimensional images; ([0052], “A contour probability map is, as known, a map which has pixel values each representing a probability of contour presence. Such a map may be computed in any know manner, for example based on a machine-learning scheme implemented on an annotated dataset of 2D images associated with contour probability maps or contour maps (a contour map being a map whose pixel values each represent binary contour presence or absence” The output of the machine-learning scheme is the probability map or contour maps, where each output value labels whether the corresponding pixel represent a contour presence or absence. This corresponds to class label, which is contour presence or contour absence.).and
insert the set of labeled points into a three-dimensional point cloud according to a plurality voting algorithm. ([0033], “The method fags within the range of structure-from-motion analysis techniques. Indeed, the method allows to reconstruct in 3D a real object based on 2D images. The 2D images are relatively simple and costless to obtain, compared to other types of signals. The method thus forms a relatively simple and costless 3D reconstruction technique.”[0035], “Now, the method relies in particular on maps having pixel values each representing a measurement of contour presence. Since the 3D modeled object optimizes an energy that rewards, for each contour map, projections of silhouette vertices of the 3D modeled object having pixel values representing a high measurement of contour presence, the method allows determining a 3D modeled object whose silhouette is projected on contours in the 2D images, to the extent possible. The method thus builds on an observation that the silhouette of a real object represented by a 2D image translates into contours in the 2D image. As a result, the 3D reconstruction allowed by the method is relatively accurate.” [0060], “The method may alternatively use a distance transform of the image in order to generate a map that contains the distance to the nearest contour at each pixel. However, such an approach requires a threshold value to identify the contours that have a probability value above that threshold and then compute the distance transform of the image. This means that the difference in probability of two contours that are both within the threshold is lost, and the contour probability information is not exploited in its entirety.” In the 3D construction process, the difference in probability of two contours that are both within the threshold is voted lost. [0061]-[0062],)
The above citations are from different embodiments/examples of State. However, It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to have combined the different embodiments/examples of State to generate a 3D point cloud from 2D images.
Regarding claim 2, State teaches:
The computer system of claim 1, wherein the two-dimensional image is captured via a drone capture device. (Biswas [0153] “The mobile device 122 may be a personal navigation device (“PND”), a portable navigation device, a mobile phone, a personal digital assistant (“PDA”), a watch, a tablet computer, a notebook computer, and/or any other known or later developed mobile device or personal computer. The mobile device 122 may also be an automobile head unit, infotainment system, and/or any other known or later developed automotive navigation system. Non-limiting embodiments of navigation devices may also include relational database service devices, mobile phone devices, car navigation devices, and navigation devices used for air or water travel.” The mobile device is used to capture images. It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to have implemented the method of State with Biswas to acquire 2D images using a drone to get images from the air.)
Regarding claim 3, State teaches:
The computer system of claim 1, the memory storing further instructions that, when executed by the one or more processors, cause the system to train the deep artificial neural network using a plurality of manually labeled training images.( State [0052], “The edge detection method described in this paper is based on the learning of a network for predicting the edges of an image, by supervised training on an image dataset where annotated the edges.” Supervised training uses user generated data (dataset where annotated the edges) to help neural network learn.)
Regarding claim 4, State teaches:
The computer system of claim 1, the memory storing further instructions that, when executed by the one or more processors, cause the system to solve a system of equations. ( State [0113], “
PNG
media_image1.png
223
890
media_image1.png
Greyscale
”)
Regarding claim 5, State teaches:
The computer system of claim 1, the memory storing further instructions that, when executed by the one or more processors, cause the system to store the trained deep artificial neural network in an electronic storage device. (State, ([0065], “The system may comprise a processor coupled to a memory and a graphical user interface (GUI), the memory having recorded thereon a computer program comprising instructions for performing the method.” State teaches the instruction are stored in memory. Trained neural network are part of the instructions that are used in State method.)
Regarding claim 6, State teaches:
The computer system of claim 1, wherein each respective set of one or more labeled points are stored in a matrix. (State, [0052], “A contour probability map is, as known, a map which has pixel values each representing a probability of contour presence. Such a map may be computed in any know manner, for example based on a machine-learning scheme implemented on an annotated dataset of 2D images associated with contour probability maps or contour maps (a contour map being a map whose pixel values each represent binarily contour presence or absence).” [0065], “The system may comprise a processor coupled to a memory and a graphical user interface (GUI), the memory having recorded thereon a computer program comprising instructions for performing the method. The memory may also store a database.”)
Regarding claim 12, State teaches:
The computer-implemented method of claim 8, further comprising: processing the set of labeled points to identify one or more tie points. (State, [0060], “The method may alternatively use a distance transform of the image in order to generate a map that contains the distance to the nearest contour at each pixel. However, such an approach requires a threshold value to identify the contours that have a probability value above that threshold and then compute the distance transform of the image. This means that the difference in probability of two contours that are both within the threshold is lost, and the contour probability information is not exploited in its entirety.”)
Claims 8-11, 13-14 recite similar limitations of claims 1-3, 5, 6-7 respectively, thus are rejected accordingly.
Claims 15-18 recite similar limitations of claims 1, 3-5 respectively, thus are rejected accordingly.
Claim 7, 14 and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over State in view of Biswas et al. (US 2020/0134833 A1).
Regarding claim 19, State teaches:
The non-transitory computer readable storage medium of claim 15, storing further executable instructions that, when executed by a processor, cause a computer to: the three-dimensional point cloud (State [0033][0035)
However, State does not, but Biswas teaches:
transmit the three-dimensional point cloud ([0043) to a user device. (Biswas, [0039]-[0040], “FIG. 1 illustrates an example system for image segmentation. In FIG. 1, one or more vehicles 124 are connected to the server 125 though the network 127. The server 125 includes an image segmentation controller 121 that identifies one or more physical objects or features in an image. The physical objects may include road objects, street furniture, road signs, or other vehicles. The road objects may include objects associated with the road that are indicative of the path of the road, including reflectors, curbs, road boundary lines, road center lines, or other objects. The street furniture may include items associated with the street including benches, traffic barriers, streetlamps, traffic lights, traffic signs, bus stops, tram stops, taxi stands, or other items. The features may include roadways, vegetation, topographical aspects, or other items. The features may be described as a shape, height, or texture. The features may include a portion of an object or characteristic of an object. Example portions of objects may include a particular side or surface of any of these objects. Example object characteristics may include a dimension of the object, a size of the object, a color of the object, a shape of the object, or other characteristics. The vehicles 124 may be directly connected to the server 125 or through an associated mobile device 122. A map developer system, including the server 125 and a geographic database 123, exchanges (e.g., receives and sends) data from the vehicles 124. The mobile devices 122 may include local databases corresponding to a local map, which may be modified according to the server 125.” )
State teaches generating data results. Biswas teaches all client server environment, where the functions can be performed by a server, the result is presented to the client.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to have implemented the method of State in a server of Biswas to take advantage of the computation power of a server and get computation result fast.
Regarding claim 20, State teaches:
The computer system of claim 1, wherein
However, State does not explicitly teach, but Biswas teaches:
The two-dimensional image corresponding to an outdoor scene including one or more outdoor objects; (FIG. 3, image 22.)
State teaches labeling image objects using AI technology. In the examples that State given, the image may not be outdoor scene. Biswas also teaches labeling image objects, which image includes outdoor scene.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to have applied the teachings of State to the outdoor scenes that are given in Biswas to enable the method of State to be used in a wider area.
Claims 14 and 20 similar limitations of claim 7, thus are rejected accordingly.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YANNA WU whose telephone number is (571)270-0725. The examiner can normally be reached Monday-Thursday 8:00-5:30 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 5712722330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YANNA WU/Primary Examiner, Art Unit 2615
Prosecution Insights