DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. CN 202311370231.7, filed on 10/20/2023.
Drawings
The drawings are objected to because portions of the figures include typos, such as Figure 1 citing “a camera disposedmounted on a removablemovable device” in box 110 and “corresponding to at least someat least partial pixels in the image” in box 120. Such typos are present throughout the drawings. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claim 2-3 and 15-16 are objected to because of the following informalities: a “preset reference-plane height value” may not comprise of multiple values, and should be “preset reference-plane height values”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 19-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 19 recites the limitation "the method according to Claim 17" in line 1. There is insufficient antecedent basis for this limitation in the claim, as there is no introduction of any method in the claims from which Claim 19 depend on or ultimately depend on. For examination purposes, “the method according to Claim 17” is replaced with “the electronic device according to Claim 17”.
Claim 20 recites the limitation “execute the instruction to implement the training method” in lines 5-6. There is insufficient antecedent basis for this limitation in the claim, as there is no introduction of any training method in the claims from which Claim 20 depend on or ultimately depend on.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 4-6, 13 and 14-19 are rejected under 35 U.S.C. 103 as being unpatentable over Masoumian (GCNDepth: Self-supervised monocular depth estimation based on graph convolutional network, 2023, January) in view of Geiger (Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite, 2012) and Koch (Depth Estimation, 2016).
Regarding Claim 1, Masoumian teaches a three-dimensional visual perception method (Section 3.5: “In monocular video datasets, based on the source frame I_s and the target frame I_t, the reconstructed image I_rec can be reconstructed using the resulting depth and the 3D pose”; Figure 3 illustrates a reconstructed image being constructed from a depth map and pose information derived from an input image. Notes: the output reconstructed image is considered to be three-dimensional visual perception, since it uses depth information and pose information (three-dimensional information) for the purpose of perceiving a scene (output image)) comprising:
Obtaining an image captured by a camera (Section 4.1: “KITTI dataset is a vision dataset for depth and poses estimation. The dataset contains 200 videos of street scenes in the daylight captured by RGB cameras”);
Determining, based on a camera parameter corresponding to the image, position information respectively corresponding to at least partial pixels in the image (Introduction: “We propose a novel autoencoder (CNN-GCN) for monocular depth estimation, which its encoder network is based on ResNet [22] as a backbone to extract key features of the input frame. A decoder network then utilizes the structure of the GCN through the whole decoding process to improve the accuracy of depth maps by learning the nodes (i.e., pixels) representation via constructing the depth maps via iteratively propagating neighbor’s information until reaching a stable point”. Notes: camera parameter, in its broadest reasonable interpretation, is any parameter related to the camera. Parameters such as camera focal length, lens distortion, and other inherent parameters to cameras necessarily correspond to the image taken by the camera);
Generating a position encoding feature map based on the position information respectively corresponding to the at least partial pixels (Figure 3 illustrates the model; Section 3.3.1: “For the encoder network, the input is an image represented as grid-like data, which is regular, and its pixels have the same amount of neighbors. CNNs can exploit the local connectivity and global structure of image data by extracting meaningful local features shared within the input images used during the training stage. Therefore, in our case, CNNs are suitable for extracting global-based visual features from the whole scene shown in the input image”);
Generating a fusion feature map based on the image and the position encoding feature map (Section 3.4: “The output of PoseNet is the relative pose between the source and target images. Afterward, a warping process, as proposed in [14], is applied for finding the corresponding pixels in the adjacent frames through the estimated depth map of the source frame and the camera ego-motion vector, and then synthesize the target frame. These two main networks provide geometry information to provide point-to-point correspondences of the reconstructed image. The whole architecture of our model is illustrated in Fig. 3”; Refer to Figure 3 for a visualization. Notes: the warping process is the act of generating a fusion feature map, where the warping process fuses the feature map from the pose encoder with the depth map); and
Generating, based on the fusion feature map, a three-dimensional visual perception result corresponding to the image by using a three-dimensional visual perception model (Section 3.4: “The output of PoseNet is the relative pose between the source and target images. Afterward, a warping process, as proposed in [14], is applied for finding the corresponding pixels in the adjacent frames through the estimated depth map of the source frame and the camera ego-motion vector, and then synthesize the target frame. These two main networks provide geometry information to provide point-to-point correspondences of the reconstructed image. The whole architecture of our model is illustrated in Fig. 3”; Refer to Figure 3 for a visualization, where the output reconstructed image is the three-dimensional visual perception result).
Masoumian does not explicitly teach obtaining an image captured by a camera mounted on a movable device.
However, Geiger teaches obtaining an image captured by a camera mounted on a movable device (Section 2.1: “We mounted all our cameras (i.e., two units, each com posed of a color and a grayscale camera) on top of our vehicle”)
Masoumian and Geiger are considered analogous in the art with respect to street centric images. While Masoumian does not explicitly teach that the images are captured by a camera mounted on a movable device, the dataset utilized by Masoumian is the dataset developed by Geiger, where Geiger states that the images are captured using vehicle mounted cameras, which are movable. It is common in the art to utilize cameras mounted on vehicles for the purpose of obtaining street centric images for use in autonomous driving, as is evident in Geiger.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that the images of Masoumian are captured via the vehicle mount method of Geiger; Doing so would yield the predictable result of street centric images from the perspective or location of a vehicle capable of movement.
Masoumian as modified does not explicitly teach that the position information respectively corresponding to at least partial pixels in the image are within a camera coordinate system.
However, Koch teaches that the position information respectively corresponding to at least partial pixels in the image are within a camera coordinate system (Background: “Depth estimation is one of the fundamental computer vision tasks, as it involves the inverse problem of reconstructing the three-dimensional scene structure from two-dimensional projections. Given a 2D image of a 3D scene, the goal of depth estimation is to recover, for every image pixel, the distance from the camera center to the nearest 3D scene point along the pixel’s viewing direction. The resulting 2D array of distance values is called the depth map, which is aligned with the camera coordinate system”).
Masoumian as modified and Koch are considered analogous in the art with respect to depth estimation. While Masoumian as modified does not explicitly state that the position information respectively corresponding to at least partial pixels in the image are within a camera coordinate system, Koch explicitly states that position information corresponding to pixels in an image are within a camera coordinate system. It is well known in the art to correlate position information of pixels in an image within the camera coordinate system, as doing so enables depth estimation, as is evident in Koch.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the position information corresponding with pixels in an image of Masoumian as modified with position information corresponding with pixels being within a camera coordinate system of Koch; Doing so would yield the predictable result of enabling depth estimation via the camera coordinate system.
Regarding Claim 4, the method according to Claim 1 is rejected over Masoumian as modified.
Masoumian as modified teaches the method wherein the generating a fusion feature map based on the image and the position encoding feature map (Masoumian, Section 3.3.1: “For the encoder network, the input is an image represented as grid-like data, which is regular, and its pixels have the same amount of neighbors. CNNs can exploit the local connectivity and global structure of image data by extracting meaningful local features shared within the input images used during the training stage. Therefore, in our case, CNNs are suitable for extracting global-based visual features from the whole scene shown in the input image”; Masoumian, Section 3.4: “The output of PoseNet is the relative pose between the source and target images. Afterward, a warping process, as proposed in [14], is applied for finding the corresponding pixels in the adjacent frames through the estimated depth map of the source frame and the camera ego-motion vector, and then synthesize the target frame. These two main networks provide geometry information to provide point-to-point correspondences of the reconstructed image. The whole architecture of our model is illustrated in Fig. 3”; Refer to Masoumian, Figure 3 for a visualization. Notes: the warping process is the act of generating a fusion feature map, where the warping process fuses the feature map from the pose encoder with the depth map. The fusion feature map is a result of the output of DepthNet, illustrated in Masoumian, Figure 3) comprises:
Generating, based on the image, a first intermediate feature map by using a first-subnetwork in a feature extraction network in the three-dimensional visual perception model (Masoumian, Section 3.3.1: “For the encoder network, the input is an image represented as grid-like data, which is regular, and its pixels have the same amount of neighbors. CNNs can exploit the local connectivity and global structure of image data by extracting meaningful local features shared within the input images used during the training stage. Therefore, in our case, CNNs are suitable for extracting global-based visual features from the whole scene shown in the input image”; Masoumian, Figure 2: “Our approach is to use four levels of GCN in constructing the depth images. The main components of the decoder network are ‘upconvolution’ layers, consisting of unpooling (up-sampling the feature maps, as opposed to pooling) and a transpose convolution that performs an inverse convolution operation. To accurately estimate the depth images, we apply the ‘upconvolution’ to feature maps and concatenate it with corresponding feature maps from the corresponding layers of the encoder network and an up-sampled coarser depth prediction using GCN of the previous layer. This approach helps the proposed model preserve the high-level information passed from coarser feature maps and the fine local information provided in lower-layer feature maps”; Refer to Masoumian, Figure 2 for an illustration of the output of an intermediate feature map; Masoumian, Figure 3 illustrates the multiple sub networks of the three-dimensional visual perception model);
Fusing the first intermediate feature map with the position encoding feature map to obtain the fusion feature map (Masoumian, Section 3.3.3: “The output of the last layer (i.e., ResNet-18-L4) from the pose encoder is a 512 feature map. In turn, our pose decoder contains four convolutional layers. The input of the pose decoder is the output of ResNet-18-L4”; Masoumian, Section 3.4: “The output of PoseNet is the relative pose between the source and target images. Afterward, a warping process, as proposed in [14], is applied for finding the corresponding pixels in the adjacent frames through the estimated depth map of the source frame and the camera ego-motion vector, and then synthesize the target frame. These two main networks provide geometry information to provide point-to-point correspondences of the reconstructed image. The whole architecture of our model is illustrated in Fig. 3”; Masoumian, Figure 2: “To accurately estimate the depth images, we apply the ‘upconvolution’ to feature maps and concatenate it with corresponding feature maps from the corresponding layers of the encoder network and an up-sampled coarser depth prediction using GCN of the previous layer. This approach helps the proposed model preserve the high-level information passed from coarser feature maps and the fine local information provided in lower-layer feature maps”; Refer to Masoumian Figure 3 for a visualization. Notes: the warping process is the act of generating a fusion feature map, where the warping process fuses the feature map from the pose encoder with the depth map); and
The generating, based on the fusion feature map, a three-dimensional visual perception result corresponding to the image by using a three-dimensional visual perception model (Masoumian, Figure 3 illustrates the three-dimensional visual perception model outputting a three-dimensional visual perception result) comprising:
Generating, based on the fusion feature map, a second intermediate feature map by using a second sub-network in the feature extraction network (Masoumian, Figure 3 illustrates the generation of intermediate feature maps and fusion feature maps, as well as the use of loss minimization between the reconstructed image produced from the fusion feature map and the target image to train the model, which results in generating subsequent (second) intermediate feature maps. Notes: Loss calculated from the output reconstructed image based on the fusion feature map is utilized to train the 3D visual perception model, where adjustments made to the model parameters result in minimized loss (known in the art for machine learning), where generating the second intermediate feature map (and subsequent intermediate feature maps) is performed during the training process for minimizing loss); and
Generating, based on the second intermediate feature map, the three-dimensional visual perception result corresponding to the image by using a prediction network in the three-dimensional visual perception model (Masoumian, Figure 3 illustrates the three-dimensional visual perception model using prediction to produce a reconstructed image (three-dimensional visual perception result) given intermediate feature maps (first, second, and subsequent feature maps derived from training)).
Regarding Claim 5, the method according to Claim 4 is rejected over Masoumian as modified.
Masoumian as modified teaches the method wherein the determining, based on a camera parameter corresponding to the image, position information respectively corresponding to at least partial pixels in the image within a camera coordinate system (Masoumian, Introduction: “We propose a novel autoencoder (CNN-GCN) for monocular depth estimation, which its encoder network is based on ResNet [22] as a backbone to extract key features of the input frame. A decoder network then utilizes the structure of the GCN through the whole decoding process to improve the accuracy of depth maps by learning the nodes (i.e., pixels) representation via constructing the depth maps via iteratively propagating neighbor’s information until reaching a stable point”; Koch, Background: “Depth estimation is one of the fundamental computer vision tasks, as it involves the inverse problem of reconstructing the three-dimensional scene structure from two-dimensional projections. Given a 2D image of a 3D scene, the goal of depth estimation is to recover, for every image pixel, the distance from the camera center to the nearest 3D scene point along the pixel’s viewing direction. The resulting 2D array of distance values is called the depth map, which is aligned with the camera coordinate system”. Notes: camera parameter, in its broadest reasonable interpretation, is any parameter related to the camera. Parameters such as camera focal length, lens distortion, and other inherent parameters to cameras necessarily correspond to the image taken by the camera) comprises:
Determining a proportional relationship between an output size supported by the first sub-network and an image size of the image (Masoumian, Figure 2 illustrates the processing and upsampling performed on the input image; Masoumian, Section 3.3.1: “For the encoder network, the input is an image represented as grid-like data, which is regular”; Masoumian, Section 3.3.2: “Regarding the depth decoder and for large-scale depth estimation, we aim to use a geometric DL network that can help extract object-based location features and keep the relationships between nodes in the resulting depth maps by generating a topological depth graph in multi-scale. Therefore, we used multi-scale GCN as shown in Fig. 2”. Notes: the output size of an encoder-decoder model such as that of Masoumian, Figure 2 is inherently a proportional relationship, which is well known in the art of machine learning regarding image processing);
Performing pixel-sampling on the image in accordance with a sampling parameter adapted to the proportional relationship, to obtain a sampling result (Masoumian, Figure 2 demonstrates upsampling performed with regards to a proportional relationship, with subsequent sampling results per layer; Masoumian, Section 3.3.1: “For the encoder network, the input is an image represented as grid-like data, which is regular”. Notes: the grid-like data is per pixel); and
Determining, based on the camera parameter corresponding to the image, position information corresponding to each pixel in the sampling result within the camera coordinate system (Masoumian, Introduction: “We propose a novel autoencoder (CNN-GCN) for monocular depth estimation, which its encoder network is based on ResNet [22] as a backbone to extract key features of the input frame. A decoder network then utilizes the structure of the GCN through the whole decoding process to improve the accuracy of depth maps by learning the nodes (i.e., pixels) representation via constructing the depth maps via iteratively propagating neighbor’s information until reaching a stable point”; Koch, Background: “Depth estimation is one of the fundamental computer vision tasks, as it involves the inverse problem of reconstructing the three-dimensional scene structure from two-dimensional projections. Given a 2D image of a 3D scene, the goal of depth estimation is to recover, for every image pixel, the distance from the camera center to the nearest 3D scene point along the pixel’s viewing direction. The resulting 2D array of distance values is called the depth map, which is aligned with the camera coordinate system”. Notes: camera parameter, in its broadest reasonable interpretation, is any parameter related to the camera. Parameters such as camera focal length, lens distortion, and other inherent parameters to cameras necessarily correspond to the image taken by the camera)
Regarding Claim 6, the method according to Claim 4 is rejected over Masoumian as modified.
Masoumian as modified teaches the method wherein the fusing the first intermediate feature map with the position encoding feature map to obtain the fusion feature map (Masoumian, Section 3.3.1: “For the encoder network, the input is an image represented as grid-like data, which is regular, and its pixels have the same amount of neighbors. CNNs can exploit the local connectivity and global structure of image data by extracting meaningful local features shared within the input images used during the training stage. Therefore, in our case, CNNs are suitable for extracting global-based visual features from the whole scene shown in the input image”; Masoumian, Figure 2: “To accurately estimate the depth images, we apply the ‘upconvolution’ to feature maps and concatenate it with corresponding feature maps from the corresponding layers of the encoder network and an up-sampled coarser depth prediction using GCN of the previous layer. This approach helps the proposed model preserve the high-level information passed from coarser feature maps and the fine local information provided in lower-layer feature maps”; Refer to Figure 3 for a visualization. Notes: the warping process is the act of generating a fusion feature map, where the warping process fuses the feature map from the pose encoder with the depth map) comprises:
Converting the position encoding feature map from an explicit representation to an implicit representation to obtain a third intermediate feature map (Masoumian, Figure 3 Clearly illustrates the conversion of the encoding feature map from an explicit representation to a more implicit representation);
Performing a convolution operation on the fourth intermediate feature map to obtain a fifth intermediate feature map (Masoumian, Figure 2: “To accurately estimate the depth images, we apply the ‘upconvolution’ to feature maps and concatenate it with corresponding feature maps from the corresponding layers of the encoder network and an up-sampled coarser depth prediction using GCN of the previous layer. This approach helps the proposed model preserve the high-level information passed from coarser feature maps and the fine local information provided in lower-layer feature maps. Each step increases the resolution twice. This process is repeated four times, providing a predicted depth map, which is half of the input image. This loop cycle is called multi-scale because, in each layer of our decoder network, the GCN is updated and up-sampled, and is sent to the next layer”); and
Performing a size adjustment on the fifth intermediate feature map to obtain the fusion feature map with a size consistent with that of the first intermediate feature map (Masoumian, Figure 2: “This process is repeated four times, providing a predicted depth map, which is half of the input image”; Masoumian, Figure 3 illustrates an output image corresponding with an intermediate feature map of a size consistent with the first intermediate feature map obtained from the input image)
Masoumian as modified does not explicitly teach overlaying the first intermediate feature map and the third intermediate feature map along a channel direction to obtain a fourth intermediate feature map, nor does it explicitly teach performing a convolution operation on the fourth intermediate feature map to obtain a fifth intermediate feature map.
However, Masoumian implicitly teaches overlaying the first intermediate feature map and the third intermediate feature map along a channel direction to obtain a fourth intermediate feature map (Masoumian, Figure 2: “To accurately estimate the depth images, we apply the ‘upconvolution’ to feature maps and concatenate it with corresponding feature maps from the corresponding layers of the encoder network and an up-sampled coarser depth prediction using GCN of the previous layer. This approach helps the proposed model preserve the high-level information passed from coarser feature maps and the fine local information provided in lower-layer feature maps. Each step increases the resolution twice. This process is repeated four times, providing a predicted depth map, which is half of the input image. This loop cycle is called multi-scale because, in each layer of our decoder network, the GCN is updated and up-sampled, and is sent to the next layer”; Masoumian, Figure 2 clearly illustrates that features in channels (the blue, green, and orange squares) are preserved when applying upconvolution (upsampling))
It is well known within the art of machine learning that combining feature maps along channel directions is performed for deriving a concatenated feature map while preserving features, as is evident in Masoumian, Figure 2.
Therefore, it would have been obvious to a person having ordinary skill in the art that overlaying a first and third feature map along a channel direction to obtain a fourth feature map is performed for producing a representation of the first and third feature map while preserving associated features.
Regarding Claim 13, Masoumian teaches a training method for a three-dimensional visual perception method (Section 3.5: “In monocular video datasets, based on the source frame I_s and the target frame I_t, the reconstructed image I_rec can be reconstructed using the resulting depth and the 3D pose”; Figure 3 illustrates a reconstructed image being constructed from a depth map and pose information derived from an input image; Refer to Figure 3 illustrating the model architecture; the minimization of loss, as apparent in Figure 3, demonstrates that the model is trained. Notes: the output reconstructed image is considered to be three-dimensional visual perception, since it uses depth information and pose information (three-dimensional information) for the purpose of perceiving a scene (output image))
Obtaining a training image captured by a camera comprising environmental information (Section 4.1: “KITTI dataset is a vision dataset for depth and poses estimation. The dataset contains 200 videos of street scenes in the daylight captured by RGB cameras”. Notes: any image used for training is a training image; likewise, any component associated with the use of the training image and the raining process is inherently a training variant);
Determining, based on a camera parameter corresponding to the training image, training position information respectively corresponding to at least partial training pixels in the training image (Introduction: “We propose a novel autoencoder (CNN-GCN) for monocular depth estimation, which its encoder network is based on ResNet [22] as a backbone to extract key features of the input frame. A decoder network then utilizes the structure of the GCN through the whole decoding process to improve the accuracy of depth maps by learning the nodes (i.e., pixels) representation via constructing the depth maps via iteratively propagating neighbor’s information until reaching a stable point”. Notes: camera parameter, in its broadest reasonable interpretation, is any parameter related to the camera. Parameters such as camera focal length, lens distortion, and other inherent parameters to cameras necessarily correspond to the image taken by the camera);
Generating a training position encoding feature map based on the training position information respectively corresponding to the at least partial training pixels (Figure 3 illustrates the model; Section 3.3.1: “For the encoder network, the input is an image represented as grid-like data, which is regular, and its pixels have the same amount of neighbors. CNNs can exploit the local connectivity and global structure of image data by extracting meaningful local features shared within the input images used during the training stage. Therefore, in our case, CNNs are suitable for extracting global-based visual features from the whole scene shown in the input image”);
Generating a training fusion feature map based on the training image and the training position encoding feature map (Section 3.4: “The output of PoseNet is the relative pose between the source and target images. Afterward, a warping process, as proposed in [14], is applied for finding the corresponding pixels in the adjacent frames through the estimated depth map of the source frame and the camera ego-motion vector, and then synthesize the target frame. These two main networks provide geometry information to provide point-to-point correspondences of the reconstructed image. The whole architecture of our model is illustrated in Fig. 3”; Refer to Figure 3 for a visualization. Notes: the warping process is the act of generating a fusion feature map, where the warping process fuses the feature map from the pose encoder with the depth map); and
Generating, based on the training fusion feature map, a training three-dimensional visual perception result corresponding to the training image by using a to-be-trained three-dimensional visual perception model (Section 3.4: “The output of PoseNet is the relative pose between the source and target images. Afterward, a warping process, as proposed in [14], is applied for finding the corresponding pixels in the adjacent frames through the estimated depth map of the source frame and the camera ego-motion vector, and then synthesize the target frame. These two main networks provide geometry information to provide point-to-point correspondences of the reconstructed image. The whole architecture of our model is illustrated in Fig. 3”; Refer to Figure 3 for a visualization, where the output reconstructed image is the three-dimensional visual perception result);
Performing information annotation on the training image to obtain annotated data associated with a three-dimensional task associated with a three-dimensional visual perception task (Figure 3 illustrates a training image annotated as a target image that the three-dimensional visual perception task compares its output to);
Training the to-be-trained three-dimensional visual perception model by using an error between the training three-dimensional visual perception result and the annotated data (Figure 3 illustrates the three-dimensional visual perception model output image is compared with the target image, where training occurs until loss is minimized (L); Section 3.5: “In monocular video datasets, based on the source frame and the target frame , the reconstructed image can be reconstructed using the resulting depth and the 3D pose. The total loss for the whole network contains three main losses, which penalizes the losses between reconstructed and target images on one side and the resulting depth and the source image on the other side” Notes: annotation, in its broadest reasonable interpretation, is the labeling of data (image data) for training purposes); and determining the trained to-be-trained three-dimensional visual perception model as a three-dimensional visual perception model in response to that the trained to-be-trained three-dimensional visual perception model meets a preset training termination condition (Section 3.6: “We implemented our method by using the PyTorch framework [44], and the proposed model was trained for 20 epochs with a batch size of 10 with one GTX 1080-TI GPU. The Adam Optimizer [45] has been utilized with an initial learning rate of and reduced by half after of the total iterations. The pre-trained ResNet-18 and ResNet-50 layers are used for the PoseNet and DepthNet encoders, respectively [46].”)
Masoumian does not explicitly teach obtaining a training image captured by a camera mounted on a movable device that captures the surrounding environment.
However, Geiger teaches obtaining a training image of the surrounding environment captured by a camera mounted on a movable device (Section 2.1: “We mounted all our cameras (i.e., two units, each com posed of a color and a grayscale camera) on top of our vehicle”; refer to Figure 1 which illustrates that the way the camera is mounted captures the environment around the movable device (vehicle), and is therefore capturing images of the surrounding environment. Notes: The images captured are used for training, and are hence training images)
Masoumian and Geiger are considered analogous in the art with respect to street centric images. While Masoumian does not explicitly teach that the images are captured by a camera mounted on a movable device, the dataset utilized by Masoumian is the dataset developed by Geiger, where Geiger states that the images are captured using vehicle mounted cameras, which are movable. It is common in the art to utilize cameras mounted on vehicles for the purpose of obtaining street centric images for use in autonomous driving, as is evident in Geiger.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that the training images of Masoumian are captured via the vehicle mount method of Geiger; Doing so would yield the predictable result of street centric images from the perspective or location of a vehicle capable of movement.
Masoumian as modified does not explicitly teach that the training position information respectively corresponding to at least partial training pixels in the image are within a camera coordinate system.
However, Koch teaches that the position information respectively corresponding to at least partial pixels in the image are within a camera coordinate system (Background: “Depth estimation is one of the fundamental computer vision tasks, as it involves the inverse problem of reconstructing the three-dimensional scene structure from two-dimensional projections. Given a 2D image of a 3D scene, the goal of depth estimation is to recover, for every image pixel, the distance from the camera center to the nearest 3D scene point along the pixel’s viewing direction. The resulting 2D array of distance values is called the depth map, which is aligned with the camera coordinate system”. Notes: whether or not the position information and partial pixels correspond to a training image is irrespective to the presence of the position information within a camera coordinate system).
Masoumian as modified and Koch are considered analogous in the art with respect to depth estimation. While Masoumian as modified does not explicitly state that the position information respectively corresponding to at least partial pixels in the image are within a camera coordinate system, Koch explicitly states that position information corresponding to pixels in an image are within a camera coordinate system. It is well known in the art to correlate position information of pixels in an image within the camera coordinate system, as doing so enables depth estimation, as is evident in Koch.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the training position information corresponding with training pixels in a training image of Masoumian as modified with position information corresponding with pixels being within a camera coordinate system of Koch; Doing so would yield the predictable result of enabling depth estimation via the camera coordinate system for training images.
Claim 14, being similar in scope to Claim 1, is rejected under the same rationale.
Claim 15, being similar in scope to Claim 2, is rejected under the same rationale.
Claim 16, being similar in scope to Claim 3, is rejected under the same rationale.
Claim 17, being similar in scope to Claim 4, is rejected under the same rationale.
Claim 18, being similar in scope to Claim 5, is rejected under the same rationale.
Claim 19, being similar in scope to Claim 6, is rejected under the same rationale.
Claims 2-3, 8-9, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Masoumian (GCNDepth: Self-supervised monocular depth estimation based on graph convolutional network, 2023, January) in view of Geiger (Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite, 2012) and Koch (Depth Estimation, 2016), and in further view of Stack Overflow (What is the difference between disparity and depth?, 2020), Cai (US 20210358153 A1) and Funes Mora (US 20230024396 A1).
Regarding Claim 2, the method according to Claim 1 is rejected over Masoumian as modified.
Masoumian as modified teaches determining, based on a camera parameter corresponding to the image, position information respectively corresponding to at least partial pixels in the image within a camera coordinate system (Masoumian, Introduction: “We propose a novel autoencoder (CNN-GCN) for monocular depth estimation, which its encoder network is based on ResNet [22] as a backbone to extract key features of the input frame. A decoder network then utilizes the structure of the GCN through the whole decoding process to improve the accuracy of depth maps by learning the nodes (i.e., pixels) representation via constructing the depth maps via iteratively propagating neighbor’s information until reaching a stable point”; Koch, Background: “Depth estimation is one of the fundamental computer vision tasks, as it involves the inverse problem of reconstructing the three-dimensional scene structure from two-dimensional projections. Given a 2D image of a 3D scene, the goal of depth estimation is to recover, for every image pixel, the distance from the camera center to the nearest 3D scene point along the pixel’s viewing direction. The resulting 2D array of distance values is called the depth map, which is aligned with the camera coordinate system”. Notes: camera parameter, in its broadest reasonable interpretation, is any parameter related to the camera. Parameters such as camera focal length, lens distortion, and other inherent parameters to cameras necessarily correspond to the image taken by the camera).
Masoumian as modified does not explicitly teach determining target depth values respectively corresponding to the at least partial pixels in the image by using a camera intrinsic parameter and a camera extrinsic parameter corresponding to the image.
However, Stack Overflow teaches determining target depth values respectively corresponding to the at least partial pixels in the image by using a camera intrinsic parameter and a camera extrinsic parameter corresponding to the image (Harshit Kumar: “The depth (the actual z location of 3d point) can be calculated by using the disparity of the corresponding point e.g. in simple cases, as follows: depth = (baseline * focal length) / disparity … where baseline is the distance b/w the cameras. By getting the depth of every pixel, you get the depth map/image”. Notes: baseline is an extrinsic camera parameter, and focal length is an intrinsic camera parameter).
Masoumian as modified and Stack Overflow are considered analogous in the art with respect to depth estimation in an image. A common method to estimate depth in an image via intrinsic and extrinsic camera parameters, as is evident in Stack Overflow. One would be motivated to utilize intrinsic and extrinsic camera parameters to estimate depth because doing so is a well-established method that accurately estimates depth.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the estimation of depth via the position information corresponding with pixels in an image within a camera coordinate system of Masoumian with the use of intrinsic and extrinsic camera parameters for estimating depth of Stack Overflow; Doing so would yield the predictable result of accurate estimation of depth in an image.
Masoumian as modified does not explicitly teach a preset-plane height value in a preset coordinate system corresponding to the movable device.
However, Cai teaches a preset-plane height value in a preset coordinate system corresponding to the movable device (Paragraph [0085]: “In the embodiments of the present disclosure, the orientation information can be a value of an included angle between a target plane set on the three-dimensional bounding box and a preset reference plane. FIG. 8 shows a top view of an image under detection. FIG. 8 includes a target plane 81 set on the three-dimensional bounding box corresponding to the object under detection and a preset reference plane 82 (the reference plane can be the plane where the camera device is located)”. Notes: the preset reference plane necessarily has a value when defined within a 3D coordinate system).
Masoumian as modified and Cai are considered analogous in the art with respect to depth information estimation. A common motivation in in depth estimation is to utilize a reference plane to aid in estimating values for depth, as is evident in Cai.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the depth estimation of Masoumian as modified with the use of a preset-plane height value of Cai; Doing so would yield the predictable result of aiding depth estimation via a reference plane.
Masoumian as modified does not explicitly teach determining the position information respectively corresponding to the at least partial pixels within the camera coordinate system by suing the camera coordinate system by using the camera intrinsic parameter and the target depth values respectively corresponding to the at least partial pixels.
However, Funes Mora teaches Determining the position information respectively corresponding to the at least partial pixels within the camera coordinate system by using the camera intrinsic parameter and the target depth values respectively corresponding to the at least partial pixels (Paragraph [0053]: “The video stream of data from each RGB-D camera thus comprises a series of RGB frames that indicates for each pixel the correspondent color and depth frames that indicates for each pixel the measured distance between the camera to the environment. Knowing the camera intrinsic parameters: width and height (the number of rows and columns in the image, respectively), the focal length of the image, the center of and the distortion model, it is possible to deproject the 2D pixel location using the correspondent depth value on the stream of images, to a 3D coordinate in the coordinate system of the camera”).
Masoumian as modified and Funes Mora are considered analogous in the art with respect to depth values corresponding to pixels in an image. It is well known in the art that camera intrinsic parameters and corresponding depth values are utilized to determine position information in camera coordinates, as is evident in Funes Mora.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the determination of position information corresponding with pixels in an image within a camera coordinate system of Masoumian with the use of camera intrinsic parameters and target depth values of Funes Mora for performing the same task; Doing so would yield the predictable result of accurately determined position information corresponding with pixels in an image within a camera coordinate system.
Regarding Claim 3, the method of Claim 2 is rejected over Masoumian as modified.
Masoumian as modified teaches the preset reference-plane height value (Cai, Paragraph [0085]: “In the embodiments of the present disclosure, the orientation information can be a value of an included angle between a target plane set on the three-dimensional bounding box and a preset reference plane. FIG. 8 shows a top view of an image under detection. FIG. 8 includes a target plane 81 set on the three-dimensional bounding box corresponding to the object under detection and a preset reference plane 82 (the reference plane can be the plane where the camera device is located)”. Notes: the preset reference plane necessarily has a value when defined within a 3D coordinate system), and
determining target depth values respectively corresponding to the at least partial pixels in the image by using a camera intrinsic parameter and a camera extrinsic parameter corresponding to the image (Stack Overflow, Harshit Kumar: “The depth (the actual z location of 3d point) can be calculated by using the disparity of the corresponding point e.g. in simple cases, as follows: depth = (baseline * focal length) / disparity … where baseline is the distance b/w the cameras. By getting the depth of every pixel, you get the depth map/image”. Notes: baseline is an extrinsic camera parameter, and focal length is an intrinsic camera parameter);
a preset reference-plane height value in a preset coordinate system corresponding to the movable device (Cai, Paragraph [0085]: “In the embodiments of the present disclosure, the orientation information can be a value of an included angle between a target plane set on the three-dimensional bounding box and a preset reference plane. FIG. 8 shows a top view of an image under detection. FIG. 8 includes a target plane 81 set on the three-dimensional bounding box corresponding to the object under detection and a preset reference plane 82 (the reference plane can be the plane where the camera device is located)”; Geiger, Section 2.1: “We mounted all our cameras (i.e., two units, each com posed of a color and a grayscale camera) on top of our vehicle”. Notes: since the camera is established to be mounted on the vehicle, the preset reference plane corresponding with the camera necessarily corresponds with the movable device) comprises:
for any target pixel in the at least partial pixels, determining a first reference depth value corresponding to the target pixel by using the camera intrinsic parameter and the camera extrinsic parameter (Stack Overflow, Harshit Kumar: “The depth (the actual z location of 3d point) can be calculated by using the disparity of the corresponding point e.g. in simple cases, as follows: depth = (baseline * focal length) / disparity … where baseline is the distance b/w the cameras. By getting the depth of every pixel, you get the depth map/image”. Notes: baseline is an extrinsic camera parameter, and focal length is an intrinsic camera parameter)
with a constraint condition that in the preset coordinate system corresponding to the movable device, a height value of a spatial point corresponding to the target pixel is the preset reference-plane height value (Cai, Paragraph [0085]: “In the embodiments of the present disclosure, the orientation information can be a value of an included angle between a target plane set on the three-dimensional bounding box and a preset reference plane. FIG. 8 shows a top view of an image under detection. FIG. 8 includes a target plane 81 set on the three-dimensional bounding box corresponding to the object under detection and a preset reference plane 82 (the reference plane can be the plane where the camera device is located)”; Geiger, Section 2.1: “We mounted all our cameras (i.e., two units, each com posed of a color and a grayscale camera) on top of our vehicle”; Koch, Triangulation: “The spatial triangulation plane or epipolar plane, which is constructed by the camera baseline b and the viewing ray sl of one camera, intersects the image plane of the other camera in a line, the epipolar line. Hence the correspondence search is confined to this line and called disparity estimation. Disparity estimation is further simplified if the epipolar line corresponds to horizontal image scan lines. This is the case in standard stereo geometry where both cameras are aligned in identical orientation and shifted in horizontal scan line direction only. Stereo cameras with convergent configurations can be rectified to a virtual alignment by a rectifying homography or with other, more general transformations [16]. A generalization to rectification is the plane sweep, where the images of multiple calibrated cameras are compared by projecting onto a common 3D reference plane”. Notes: the preset reference plane necessarily has a value when defined within a 3D coordinate system. As the common reference plane is utilized in determining disparity, which is utilized in determining depth in an image, a height value of a spatial point corresponding to the target pixel is the preset reference-plane height value via projecting onto the plane).
Masoumian as modified does not explicitly teach that the preset reference-plane height value may be either a preset sky-plane height value or a preset ground-plane height value, nor does it teach determining a reference depth value for both values, where the height value of the spatial point corresponds to the target pixel, and determining the target depth value corresponding to the target pixel based on the smaller of the two reference depth values.
However, it is well known in the art that preset reference-planes may be any such plane within an image from which depth is estimated against; the sky and the ground are common planes utilized as reference-planes. Therefore, it would have been obvious to a person having ordinary skill in the art that the reference-plane used for depth estimation may correspond with the height of either the sky-plane or ground-plane.
A person having ordinary skill in the art would also have considered determining a target depth value corresponding to the target pixel based on how small the reference depth value is for one reference plane (such as the sky) and a second reference plane (such as the ground); A common motivation in the art is to utilize effective reference planes that allow accurate depth estimations. If the reference depth value is larger, there is a more significant margin for error in depth prediction of a pixel when compared to a smaller reference depth value; Hence, a person having ordinary skill in the art would be motivated to minimize the reference depth value obtained from a reference plane in an image.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the determining of reference depth values based on a reference plane corresponding to pixels in an image via intrinsic and extrinsic camera parameters of Masoumian as modified with the well-known knowledge regarding the ground and sky serving as reference planes in an image along with the motivation to minimize depth reference values via the selection of an adequate reference plane; Doing so would yield the predictable result of more accurately estimating depth values corresponding to pixels in an image.
Regarding Claim 8, the method according to Claim 2 is rejected over Masoumian as modified.
Masoumian as modified teaches the position information corresponding to any target pixel in the at least partial pixels (Masoumian, Introduction: “We propose a novel autoencoder (CNN-GCN) for monocular depth estimation, which its encoder network is based on ResNet [22] as a backbone to extract key features of the input frame. A decoder network then utilizes the structure of the GCN through the whole decoding process to improve the accuracy of depth maps by learning the nodes (i.e., pixels) representation via constructing the depth maps via iteratively propagating neighbor’s information until reaching a stable point”; Koch, Background: “Depth estimation is one of the fundamental computer vision tasks, as it involves the inverse problem of reconstructing the three-dimensional scene structure from two-dimensional projections. Given a 2D image of a 3D scene, the goal of depth estimation is to recover, for every image pixel, the distance from the camera center to the nearest 3D scene point along the pixel’s viewing direction. The resulting 2D array of distance values is called the depth map, which is aligned with the camera coordinate system”).
Masoumian as modified does not explicitly teach that the position information comprises a first coordinate value along an x-axis of the camera coordinate system, a second coordinate value along a y-axis of the camera coordinate system, and a third coordinate value along a z-axis of the camera coordinate system.
However, Funes Mora teaches a first coordinate value along an x-axis of the camera coordinate system, a second coordinate value along a y-axis of the camera coordinate system, and a third coordinate value along a z-axis of the camera coordinate system (Funes Mora, Paragraph [0053]: “The video stream of data from each RGB-D camera thus comprises a series of RGB frames that indicates for each pixel the correspondent color and depth frames that indicates for each pixel the measured distance between the camera to the environment. Knowing the camera intrinsic parameters: width and height (the number of rows and columns in the image, respectively), the focal length of the image, the center of and the distortion model, it is possible to deproject the 2D pixel location using the correspondent depth value on the stream of images, to a 3D coordinate in the coordinate system of the camera).
Masoumian as modified and Funes Mora are considered analogous in the art with respect to estimating depth via position information corresponding with pixels in an image. While Masoumian does not explicitly teach coordinates for the camera coordinate system, an x, y, and z coordinate within the camera coordinate system in which a pixel is mapped to is implicit. It would have been obvious to a person having ordinary skill in the art that the presence of a camera coordinate system entails x, y, and z coordinates associated with a pixel from the image.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the position information corresponding to pixels in an image within a camera coordinate system with the x, y, and z coordinates of a point mapped to from a pixel in an image of Funes Mora; Doing so would yield the predictable result of mapping depth in an image in a 3D coordinate system corresponding with the camera.
Masoumian as modified does not explicitly teach that the first coordinate value, second coordinate value, and third coordinate value are stored at corresponding positions of different channels in the position encoding feature map.
However, it is well known in the art that coordinate values within a camera coordinate system are preserved in depth maps, as is evident in Masoumian (refer to Massoumian, Figure 2, where the depth map corresponds with the original image, which necessarily requires coordinate information to be preserved via channels during convolution). Therefore, the x, y, and z channels are implicit.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that the preservation of the x, y, and z camera coordinate values across the performed convolutions of Masoumian implies that the x, y, and z coordinate values are stored in different channels in the position encoding feature map.
Claim 9, being similar in scope to Claim 8, is rejected under the same rationale.
Regarding Claim 20, the three-dimensional visual perception model according to Claim 8 is rejected over Masoumian as modified.
Masoumian as modified teaches an electronic device (Masoumian, Section 3.6: “We implemented our method by using the PyTorch framework ]44], and the proposed model was trained for 20 epochs with a batch size of 10 with one GTX 1080-TI GPU”. Notes: an electronic device is inherent to the utilization of a processor for training a machine learning model) wherein the electronic device comprises:
A processor (Masoumian, Section 3.6: “We implemented our method by using the PyTorch framework ]44], and the proposed model was trained for 20 epochs with a batch size of 10 with one GTX 1080-TI GPU”); and a memory (memory is inherent to training machine learning models, as memory is necessary for storing data for computing), configured to store a processor-executable instruction, wherein the processor is configured to read the executable instruction from the memory, and execute the instruction to implement the method for a three-dimensional visual perception model (Masoumian, Section 3.6: “We implemented our method by using the PyTorch framework ]44], and the proposed model was trained for 20 epochs with a batch size of 10 with one GTX 1080-TI GPU”. Notes: the method being carried out by the processor and memory is inherent to training the machine learning model).
Claims 7 and 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Masoumian (GCNDepth: Self-supervised monocular depth estimation based on graph convolutional network, 2023, January) in view of Geiger (Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite, 2012) and Koch (Depth Estimation, 2016), in further view of Funes Mora (US 20230024396 A1).
Regarding Claim 7, the method according to Claim 1 is rejected over Masoumian as modified.
Masoumian as modified teaches the position information corresponding to any target pixel in the at least partial pixels (Masoumian, Introduction: “We propose a novel autoencoder (CNN-GCN) for monocular depth estimation, which its encoder network is based on ResNet [22] as a backbone to extract key features of the input frame. A decoder network then utilizes the structure of the GCN through the whole decoding process to improve the accuracy of depth maps by learning the nodes (i.e., pixels) representation via constructing the depth maps via iteratively propagating neighbor’s information until reaching a stable point”; Koch, Background: “Depth estimation is one of the fundamental computer vision tasks, as it involves the inverse problem of reconstructing the three-dimensional scene structure from two-dimensional projections. Given a 2D image of a 3D scene, the goal of depth estimation is to recover, for every image pixel, the distance from the camera center to the nearest 3D scene point along the pixel’s viewing direction. The resulting 2D array of distance values is called the depth map, which is aligned with the camera coordinate system”).
Masoumian as modified does not explicitly teach that the position information comprises a first coordinate value along an x-axis of the camera coordinate system, a second coordinate value along a y-axis of the camera coordinate system, and a third coordinate value along a z-axis of the camera coordinate system.
However, Funes Mora teaches a first coordinate value along an x-axis of the camera coordinate system, a second coordinate value along a y-axis of the camera coordinate system, and a third coordinate value along a z-axis of the camera coordinate system (Funes Mora, Paragraph [0053]: “The video stream of data from each RGB-D camera thus comprises a series of RGB frames that indicates for each pixel the correspondent color and depth frames that indicates for each pixel the measured distance between the camera to the environment. Knowing the camera intrinsic parameters: width and height (the number of rows and columns in the image, respectively), the focal length of the image, the center of and the distortion model, it is possible to deproject the 2D pixel location using the correspondent depth value on the stream of images, to a 3D coordinate in the coordinate system of the camera).
Masoumian as modified and Funes Mora are considered analogous in the art with respect to estimating depth via position information corresponding with pixels in an image. While Masoumian does not explicitly teach coordinates for the camera coordinate system, an x, y, and z coordinate within the camera coordinate system in which a pixel is mapped to is implicit. It would have been obvious to a person having ordinary skill in the art that the presence of a camera coordinate system entails x, y, and z coordinates associated with a pixel from the image.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the position information corresponding to pixels in an image within a camera coordinate system with the x, y, and z coordinates of a point mapped to from a pixel in an image of Funes Mora; Doing so would yield the predictable result of mapping depth in an image in a 3D coordinate system corresponding with the camera.
Masoumian as modified does not explicitly teach that the first coordinate value, second coordinate value, and third coordinate value are stored at corresponding positions of different channels in the position encoding feature map.
However, it is well known in the art that coordinate values within a camera coordinate system are preserved in depth maps, as is evident in Masoumian (refer to Massoumian, Figure 2, where the depth map corresponds with the original image, which necessarily requires coordinate information to be preserved via channels during convolution). Therefore, the x, y, and z channels are implicit.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that the preservation of the x, y, and z camera coordinate values across the performed convolutions of Masoumian implies that the x, y, and z coordinate values are stored in different channels in the position encoding feature map.
Claim 10, being similar in scope to Claim 7, is rejected under the same rationale.
Claim 11, being similar in scope to Claim 7, is rejected under the same rationale.
Claim 12, being similar in scope to Claim 7, is rejected under the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYMOND CHUN LAM LI whose telephone number is (571)272-5124. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYMOND CHUN LAM LI/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights