DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
[Claim 6] “…a data obtaining module…” (means) + “…configured to obtain a point cloud set and an object segmentation result of a target scene…” (function)
[Claim 6] “…an object point cloud subset determining module…” (means) + “…configured to determine a point cloud subset of each object according to the point cloud set and the object segmentation result…” (function)
[Claim 6] “…a scene graph prediction module…” (means) + “configured to determine a 3DSSG of the target scene according to a point cloud subset of any object and object auxiliary information by using a 3DSSG prediction model” (function)
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
“…a data obtaining module…” [no corresponding structure described in the specification]
“…an object point cloud subset determining module…” [no corresponding structure described in the specification]
“…a scene graph prediction module…” [no corresponding structure described in the specification]
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 6 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Claim 6 contains limitations that invoke 35 U.S.C. 112(f). There is no corresponding structure present within the specification of the application.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 6 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
With respect to claim 6, claim limitations “…a data obtaining module…”, “…an object point cloud subset determining module…”, and “…a scene graph prediction module…” invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The disclosure is devoid of any associated structure that performs the function present in the claim limitations. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 12-16 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claims 12-16 expand (broaden) the definition of memory. Applicant may cancel the claims, amend the claims to place the claims in proper dependent form, rewrite the claims in independent form, or present a sufficient showing that the dependent claims comply with the statutory requirements.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 12-16 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims do not fall within at least one of the four categories of patent eligible subject matter because they are directed to non-transitory forms of signal transmission (signals per se).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6-7 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Wei (CN 114266863 A) in view of Wu (CN 114283294 A) and Liu (CN 114743123 A).
With respect to claim 1, A three-dimensional semantic scene graph (3DSSG) generation method, comprising: obtaining a point cloud set (“extracting the basic point cloud feature from the 3 D scene graph data set;” page 3 paragraph 4) and an object segmentation result of a target scene (“performing point cloud target detection to generate a 3 D boundary frame” page 3 paragraph 4), wherein the target scene comprises multiple objects (“Geometries include support, on, on, off, in, out, in, out, in, out and in, in order to describe the relative positional relationship between objects” page 2 paragraph 6);
determining a point cloud subset of each object according to the point cloud set and the object segmentation result (“Further, the basic point cloud feature and the geometric attribute value of the 3 D boundary frame is input; for each 3 D boundary frame, firstly inquiring all points in the boundary frame, then ordering the points according to their distance to the boundary frame centre” page 3 paragraph 3);
and determining a 3DSSG of the target scene according to a point cloud subset of any object and object auxiliary information (“generating the 3 D scene graph according to the base point cloud feature and the 3 D boundary frame;” page 5 paragraph 8) by using a 3DSSG prediction model (“the RoI feature, the word vector, position vector; obtaining the characteristic representation of the level of the article by a bidirectional LSTM module, then inputting the feature and the object positioning image feature to the attention module, obtaining the final characteristic representation, and then obtaining the last 3 D scene graph through a softmax” page 6 paragraph 4), wherein the object auxiliary information comprises geometric information (“the geometric parameter of the boundary frame to construct a positioning map, which is used for learning the influence of the relative position of different article pairs in the attention module. the node of the graph is the 3 D boundary frame detected by us, the side is the relative position between the boundary frame, represented by a 4-dimensional vector, the first dimension represents the relative distance, other three-dimensional representative direction” page 6 paragraph 3);
the 3DSSG prediction model is obtained by training a 3DSSG initial prediction model by using a training dataset (See Examiner figure 1) ; the training dataset is a 3DSSG dataset (see Examiner figure 1);
PNG
media_image1.png
906
542
media_image1.png
Greyscale
Examiner Figure 1: Translated Wei Figure 1
the 3DSSG prediction model comprises a Transformer-based feature extractor (“In relation prediction, the semantic category of the object is very important, because some articles are more related to the nature than other articles. In the present invention, using word to vec to code the type of the object as word vector. In the 3 D space, the position of the boundary frame is also very important, so the position parameter of the object is encoded by an MLP of one 2 layer.” Page 6 paragraph 2), a first multi-layer perceptron (MLP) (“In relation prediction, the semantic category of the object is very important, because some articles are more related to the nature than other articles. In the present invention, using word to vec to code the type of the object as word vector. In the 3 D space, the position of the boundary frame is also very important, so the position parameter of the object is encoded by an MLP of one 2 layer.” Page 6 paragraph 2), and a scene graph generation module (“…then obtaining the last 3 D scene graph through a softmax” page 6 paragraph 4);
Wei does not teach wherein the object auxiliary information comprises a bounding box size of the point cloud subset, an object length, an object volume, and an object point cloud spatial distribution standard deviation (SD); the 3DSSG prediction model comprises a graph neural network (NN)-based relationship reasoning module; both the Transformer-based feature extractor and the first MLP are connected to the graph NN-based relationship reasoning module; and the graph NN-based relationship reasoning module is connected to the scene graph generation module.
Wu teaches wherein the object auxiliary information comprises a bounding box size of the point cloud subset (“for the surface information contained in the sub point cloud, the surface information can describe the cloud whole profile, such as point cloud whole extending direction, the whole shape, extreme value and area size and so on information” page 6 paragraph 5 lines 12-14), an object length (“the surface information can describe the cloud whole profile, such as point cloud whole extending direction, the whole shape, extreme value and area size and so on information page 6 paragraph 5 lines 13-14), an object volume (“the surface information can describe the cloud whole profile, such as point cloud whole extending direction, the whole shape, extreme value and area size and so on information” page 6 paragraph 5 lines 13-14), and an object point cloud spatial distribution standard deviation (SD) (“…Relative Standard Deviation (RSD for short)…” page 6 paragraph 5 lines 11-12);
Wu is analogous art in the same field of endeavor as the claimed invention. Wu is directed towards improvements to semantic scene understanding (“The three-dimensional point cloud has high precision and density, comprising rich semantic feature information gradually becoming the main data form of three-dimensional semantic scene understanding research.” Page 2 paragraph 1 And “ The invention claims an extracting method based on neural network of the cloud point cloud, system, device and storage medium, the main purpose is to not only fully and effectively describe the point cloud itself, but also can not increase the calculation amount.” Page 2 paragraph 4 and “FIG. 1 is an embodiment of the present invention provides a point cloud feature extraction method based on the neural network scene graph;” page 4 paragraph 8). A person of ordinary skill in the art, before the effective filing date of the claimed invention would have found it obvious to combine the teachings of Wei and Wu by utilizing Wu’s point cloud analysis process to further the capabilities of Wei’s scene graph generation model with the expectation that doing so would lead to improvements in point cloud feature analysis and lowering of computational power necessity (“The invention claims an extracting method based on neural network of the cloud point cloud, system, device and storage medium, firstly automatically dividing the target point cloud, capable of remove the noise point. then extracting each sub point cloud based on the geometric feature of the three-dimensional point line surface, considering the point cloud information midpoint, line and surface three aspects, comprising rich information, so as to effectively express the point cloud; and it can effectively remove the unnecessary point cloud information, reduce the data amount, reduce the point cloud computing amount, so as to reduce the requirement of the computer performance.” Page 4 paragraph 7).
Liu teaches the 3DSSG prediction model comprises a Transformer-based feature extractor (“…using the depth residual network to extract the image feature for the object in the room layout and the image, and flattening the image feature and the layout node local feature and the object node feature, respectively then connecting in series to form a vector, then using the multi-layer perceptron MLP to encode the vector into the node representation vector with the same length;” page 3 paragraph 10), a first multi-layer perceptron (MLP) (“…using independent multi-layer perceptron MLP to decode the representation vector of the node in the graph neural network…” page 4 paragraph 2), a graph neural network (NN)-based relationship reasoning module (“…scene graph convolution network…” page 4 paragraph 1), and a scene graph generation module (“the invention uses the scene graph convolution network to realize the understanding and utilization of the relation between the object and the room layout in the two stages of the image estimation and the graph optimization, the three-dimensional scene understanding result is more accurate.” Page 4 paragraph 8);
both the Transformer-based feature extractor and the first MLP are connected to the graph NN-based relationship reasoning module (“In each iteration of the image optimization, scene graph convolution network through four-step message transmission, using independent multi-layer perceptron MLP to decode the representation vector of the node in the graph neural network the corresponding object or layout bounding box parameter and implicit expression vector and object type, remaining the residual of the label, and adding to the estimation value of the initial stage to obtain the estimation result of this iteration, the node in the graph neural network an object node and a layout node;” page 4 paragraph 2);
and the graph NN-based relationship reasoning module is connected to the scene graph generation module (“scene graph convolution network outputs object implicit expression vector, three-dimensional model output grid representation model of the object by local depth implicit function decoding object, and combining the step S2 to optimize the iterative map estimation result to realize scene reconstruction.” Page 3 paragraph 2).
Liu is analogous art in the same field of endeavor as the claimed invention. Liu is directed towards scene understanding (“The invention relates to computer vision field technical field specifically to a scene understanding method based on hidden function three-dimensional representation and neural network, for realizing three dimensional scene understanding task comprising layout estimation, camera attitude estimation, three dimensional target detection, single-eye three-dimensional scene reconstruction.” Page 2 paragraph 1). A person of ordinary skill before the effective filing date of the claimed invention would have found it obvious to combine the teachings of Wei, Wu and Liu by utilizing the incorporating Liu’s machine learning model structure into the combined system of Wei and Wu, with the expectation that doing so would lead to increased accuracy in scene understanding (“the invention uses the scene graph convolution network to realize the understanding and utilization of the relation between the object and the room layout in the two stages of the image estimation and the graph optimization, the three-dimensional scene understanding result is more accurate.” Page 4 paragraph 8).
With respect to claim 6, Wei, Wu, and Liu render obvious all limitations in consideration of claim 1, due to the substantial similarities between claim 1 and claim 6, with claim 6 being directed to a system which preforms the method of claim 1.
With respect to claim 7, Wei further teaches an electronic device (“…terminal device…” page 7 paragraph 4), comprising a memory (“…The computer readable storage medium comprises built-in storage medium in the terminal device, providing a storage space, storing the operation system of the terminal, also can include the extension storage medium the terminal device supported. and storing one or more instructions adapted to be loaded and executed by the processor in the storage space…” page 7 paragraph 4) and a processor (“…The computer readable storage medium comprises built-in storage medium in the terminal device, providing a storage space, storing the operation system of the terminal, also can include the extension storage medium the terminal device supported. and storing one or more instructions adapted to be loaded and executed by the processor in the storage space…” page 7 paragraph 4), wherein the memory is configured to store a computer program (“…the computer readable storage medium a memory device in the terminal device, for storing program and data...” page 7 paragraph 4), and the processor runs the computer program to enable the electronic device to perform the 3DSSG generation method according to claim 1 (“…The processor can load and execute one or more instructions stored in the computer readable storage medium, so as to realize the corresponding step of the embodiment in the method for generating the 3 D scene graph based on point cloud.” page 7 paragraph 4)
With respect to claim 12, Wei, Wu, and Liu teach the electronic device according to claim 7. Wei further teaches wherein the memory is a readable storage medium (“…specifically using computer readable storage medium (Memory)…” page 7 paragraph 4).
Allowable Subject Matter
Claims 2-5 and 8-11 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
With respect to claim 2, Wei, Wu, and Liu teach the 3DSSG generation method according to claim 1. No prior art has been found that teaches all claim limitations. The most relevant, Phalak (US 20210279950 A1) teaches a methodology and associated ML architecture that utilizes two MLPs, downsampling and the encoding and decoding of features in sequence, but fails to teach any further limitations present in the claim. Specifically, Phalak does not teach updating the object feature and object relationship feature by employing a third MLP and AttnNet and the further described structural elements of the scene graph generation module present within the last paragraph of the claim.
With respect to claims 3-5, they depend from claim 2 and thus inherit the above described limitations.
With respect to claim 8, it is substantially similar to claim 2. Wei, Wu, and Liu teach the electronic device according to claim 7 and Phalak teaches a methodology and associated ML architecture that utilizes two MLPs, downsampling and the encoding and decoding of features in sequence, but fails to teach any further limitations present in the claim.
With respect to claims 9-11, they are substantially similar to claims 3-5 and since they depend from claim 8, they thus inherit its limitations.
With respect to claims 13-16, they are all substantially similar to each other and inherent, from claim 8, limitations which the prior art of record fails to teach, however there still remains issues in regards to subject matter eligibility (see Claim Rejections - 35 USC § 101 and 35 USC § 112)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Gubbi (US 20240020962 A1)- discloses a semantic scene graph generation method that utilizes a GNN and an edge proposal neural network
Kar (US 20200160178 A1)- discloses utilizing a GCN to create transformed scene graphs to create synthetic scene graph datasets.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REBECCA C WILLIAMS whose telephone number is (571)272-7074. The examiner can normally be reached M-F 7:30am - 4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew W Bee can be reached at (571)270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/REBECCA COLETTE WILLIAMS/Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677
Prosecution Insights