DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, and 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Huang (CN 114743084 A) in view of Jiang (CN 108596886 A).
Regarding Claim 1, Huang teaches a method for detecting an insulator fault of a transmission line based on an unfolding super-resolution network (USRNet) and an improved
performing super-resolution reconstruction on an original image through a deep USRNet, to implement optimization of a test dataset ([pg. 3, paragraph 2]: performing super-resolution reconstruction on the test set image, [pg. 4, paragraph 1]: Preferably, the super-resolution reconstruction technology used in the step three is USRNet);
based ([pg. 3, paragraph 1]: clustering the mark frame obtained in step one, to generate anchor frame suitable for detecting insulator and fault target); and changing a structure of a multi-scale feature fusion module, and introducing, at a prediction end, a detection head comprising a larger feature map, to detect a small fault target ([pg. 9, paragraph 5]: the sensitivity of the enhanced YOLOv5x to the small target, the Neck part structure of theYOLOv5x model is changed, to add a detecting head for detecting small target and corresponding three size anchor frame, [pg. 9, paragraph 6]: improvement of the invention changes the number of the convolution kernel channels…and then changes the number of the concat layer channels… so that the output feature map originally here is changed from 512 * 80 * 80 to 1024 * 80 * 80); and
optimizing overall performance of the model based on an effective intersection over union (EIOU)_Loss function ([pg. 9]: frame loss function of the original YOLOv5x is replaced. YOLOv5x is composed of three parts, respectively is classloss function BECLogits, confidence loss function BCEclsloss and frame loss function CIOU Loss, [pg.9, last paragraph into pg. 10]: using EIOU-Loss to replace the new frame loss function, the penalty term of EIOU-loss comprises overlapping loss, central distance loss, width loss three parts, the formula is as follows); and performing, based on a constructed improved ([abstract]: model for target detection, identifying the insulator and the fault target in the image and evaluating the result).
Huang does not explicitly teach using a MobileNet-SSD detection model. However, using a MobileNet-SSD detection model is well known in the art as taught by Jiang ([abstract]: invention claims an aerial image insulator based on deep learning of the dropping sheet rapid fault detection method…establishing deep learning target detecting model SSD, comprises a base network MobileNet).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified the teachings of Huang by substituting the teachings of Jiang, particularly by substituting the YOLO detection model for the MobileNet-SSD detection model. Doing so would provide the predictable result of using a detection model for fault detection.
Regarding Claim 2, the Huang and Jiang combination teaches the method for detecting an insulator fault of a transmission line based on a USRNet and an improved MobileNet-SSD algorithm according to claim 1. In addition, Huang teaches wherein the performing super-resolution reconstruction on an original image through a deep USRNet, ([pg. 4, paragraph 1]: Preferably, the super-resolution reconstruction technology used in the step three is USRNet) specifically comprises:
modelling a common problem of super-resolution:
z
k
=
a
r
g
m
i
n
Z
y
-
z
⊗
g
↓
s
2
+
µ
σ
2
z
-
x
k
-
1
2
(see equation on pg. 17, line 18 or pg. 22 equation in [0038]),
x
k
=
a
r
g
m
i
n
Z
µ
2
z
k
-
x
2
+
λ
∅
(
x
)
(see equation on pg. 17 line 19 or pg.22 equation in [0039])
wherein x represents a high-resolution image of the transmission line; z represents an auxiliary variable introduced based on a semi-quadratic splitting algorithm; g represents a fuzzy kernel;
µ
represents a penalty parameter for controlling a difference between z and x; k represents a quantity of iterations, k=1, …, 8; argmin represents a value of a subscript variable z when a posterior formula is smallest; s represents a multiple for bicubic sampling; y represents a low-resolution image of the transmission line;
⊗
represents a symbol of a tensor product;
↓
represents a subsampling operation;
∅
(
x
)
represents noise intensity;
λ
represents a hyperparameter for controlling the noise intensity; and
σ
represents a noise level ([pg.8, paragraph 5]: x is HR image, y is LR image, g is fuzzy kernel, s is double three times sampling multiple. namely the HR image after the fuzzy kernel g fuzzy processing and double three times of sampling, and the error between the corresponding original LR image is the minimum. after considering an additional prior knowledge noise intensity, obtaining: wherein the first item is data item, the second item is the prior item, φ (x) is the noise intensity, λ is the super parameter for controlling the noise intensity, σ is the noise level. In order to minimize the error E(x), USRNet uses the semi-quadratic resolution algorithm, see equation on pg. 17 or pg. 22 equations in [0038], [0039]); and
solving x and z through iteration of a neural network, to obtain a clearest high-resolution image
x
8
of the transmission line ([pg. 8, third to last paragraph]: it is possible to solve x, z optimization target by iteration).
Regarding Claim 3, the Huang and Jiang combination teaches the method for detecting an insulator fault of a transmission line based on a USRNet and an improved MobileNet-SSD algorithm according to claim 2. In addition, Huang teaches wherein a structure of the deep USRNet mainly comprises three parts;
a first part is a data module D, and is used to solve
z
k
=
a
r
g
m
i
n
Z
y
-
z
⊗
g
↓
s
2
+
µ
σ
2
z
-
x
k
-
1
2
and the data module D performs fast Fourier transform
F
(
•
)
and complex conjugate transform
F
-
1
(
•
)
through pytorch, introduces a hyperparameter
a
k
and minimizes
z
k
([pg. 8, last paragraph] : the first part of data module (Data module D) is used for solving zk, which uses pyr to realize fast Fourier transform and complex conjugate transform thereof, introducing super-parameter alpha k minimizing zk);
z
k
=
F
-
1
1
a
k
d
-
F
-
g
⊙
s
F
g
d
⇓
s
F
-
g
F
g
⇓
s
+
a
k
, wherein
F
-
(
•
)
represents a conjugate complex of
F
(
•
)
;
a
k
represents a hyperparameter;
F
(
g
)
represents Fourier transform performed on the fuzzy kernel;
⇓
represents a subsampler;
⊙
represents an XNOR operator;
d
=
F
-
g
F
y
↑
s
+
a
k
F
(
x
k
-
1
)
,
↑
represents an upsampling operation; and
a
k
=
μ
k
σ
2
(see equations on pg.22 particularly [0043], [0044], and [0045]);
the solution process is abbreviated as follows:
z
k
=
D
(
x
k
-
1
,
s
,
g
,
y
,
a
k
)
, wherein
x
0
is obtained through
y
by nearest interpolation ([pg. 9, paragraph 1]: solving process can be abbreviated as: zk = D (xk-1, s, g, y, α k) wherein x0 is obtained by the most adjacent interpolation by y, see equation on pg. 22, particularly [0047]);
a second part is a prior module
P
and is used to perform, through a U-shaped network added with a residual term, noise reduction on the original image, to solve
x
k
=
a
r
g
m
i
n
Z
µ
2
z
k
-
x
2
+
λ
∅
(
x
)
, and the noise level is as follows (see pg. 22, equation on [0039], [pg. 9] : the second part prior module (Prior module P) uses U-Net adding the residual error to solve the original image noise reduction formula xk):
β
k
=
λ
/
μ
k
(see equation on pg. 22, [0050]), wherein
the noise reduction process is abbreviated as follows:
x
k
=
P
(
z
k
,
β
k
)
(see equation on pg. 22, [0052], [pg. 9]: noise reduction process can be abbreviated as: xk = P (zk, β k)); and
a third part is a hyperparameter module
H
, and is used to calculate
a
k
and
β
k
required for each iteration:
α
,
β
=
H
(
σ
,
s
)
(see pg. 23 equation in [0054], [pg. 9, paragraph 2]: the third part of the reference module (Hyper-parameter module H) is used for calculating the α k, β k needed when each iteration,[α, β] = H (σ, s)), wherein
the hyperparameter module comprises three fully connected layers, each layer has 64 hidden nodes, an activation function of the first two layers is ReLU, and an activation function of the last layer is Softplus ([pg. 9, paragraph 3]: The module is composed of three layers of full connection layer, the hidden node number of each layer is 64, the activation function of the front two layers is ReLU, the last layer is Softplus).
Regarding Claim 4, representative of Claim 7, the Huang and Jiang combination teaches the method for detecting an insulator fault of a transmission line based on a USRNet and an improved MobileNet-SSD algorithm according to claim 1. In addition, Huang teaches wherein the performing clustering on a labeled box through K-means++, to generate an anchor box matching a target size of a fault of a transmission line comprises:
randomly taking a target box of a sample as an initial clustering center, wherein the target box of the sample is the labeled box; and calculating a minimum intersection over union IOU distance
A
(
x
)
between a remaining labeled box and a current clustering center ([pg. 3, paragraph 8]: 1) randomly selecting a sample target frame as the initial clustering centre, calculating the minimum IOU distance A (x) of the rest sample frame and the current clustering centre):
A
x
'
=
1
-
I
(
x
'
,
c
)
wherein ([pg. 3, paragraph 8]: A (x) = 1-IOU (x, c)),
I represents an intersection over union between target boxes of two samples; x’ represents a labeled box of a sub-target sample; and c represents a clustering center ([pg. 3, paragraph 9]: wherein, IOU represents the cross-to-parallel ratio between two rectangular frames, x is the sub-target mark sample frame, c represents the centre of the clustering);
calculating probability
O
x
'
that a target box of each insulator sample is taken as a next clustering center, and selecting the next clustering center through a roulette wheel method ([pg. 3, paragraph 10]: calculating the probability O (x) of each insulator sample frame selected as the next clustering centre, and selecting the next clustering centre by using the wheel disc method):
O
x
'
=
A
x
'
2
∑
x
'
∈
X
A
x
'
2
, (see pg. 16, equation on line 29 or pg. 21 equation in [0022]) wherein
X represents a total sample of the labeled box of the target ([pg. 3, paragraph 10]: wherein X is the target mark frame total sample);
repeating the foregoing steps until K clustering centers are selected ([pg. 3, paragraph 10]: 3) repeating the steps 1) and 2) until the K clustering centres are selected);
calculating a distance from each sample x’ in the dataset to the K clustering centers, wherein the sample x’ in the dataset is the high-resolution image of the transmission line, assigning the sample x’ to a category corresponding to a clustering center with a smallest distance, and recalculating a clustering center of each category
c
l
, wherein a formula is as follows; and re-updating classification and the clustering center until the size of the anchor box remains unchanged ([pg. 3, paragraph 11]: calculating the distance of each sample x to K clustering centres in the data set, and dividing the sample into the class corresponding to the minimum distance clustering centre, re-calculating the clustering centre of each class c1, repeatedly updating the classification and clustering centre until the anchor frame size is not changed):
c
l
=
1
c
l
∑
x
'
∈
c
l
x
'
, (see pg. 16 equation on line 35 or pg.21 equation in [0026]) wherein
l = 1, …, K; K represents a quantity of different sizes of anchor boxes and a value of K is determined by a quantity of anchor boxes in the ([pg. 3, last paragraph]: wherein, l= 1, ..., K, K is the number of anchor frame with different sizes, the value is determined by the number of anchor frame of the detection model).
Jiang teaches using a MobileNet-SSD as a detection model ([abstract]: invention claims an aerial image insulator based on deep learning of the dropping sheet rapid fault detection method…establishing deep learning target detecting model SSD, comprises a base network MobileNet).
Regarding Claim 6, representative of Claim 8, the Huang and Jiang combination teaches the method for detecting an insulator fault of a transmission line based on a USRNet and an improved MobileNet-SSD algorithm according to claim 1. In addition, Huang teaches wherein the optimizing overall performance of the model based on an EIOU_Loss function ([pg. 6, paragraph 2]: compared with the original frame loss function CIOU-Loss, the EIOU-Loss adopted by this invention can accelerate the convergence speed of the detection model) specifically comprises:
replacing CIOU_Losses of the original model with EIOU_Losses ([pg. 6, paragraph 2]: compared with the original frame loss function CIOU-Loss, the EIOU-Loss adopted by this invention can accelerate the convergence speed of the detection model), wherein penalty terms of EIOU_Losses comprise an overlap loss
L
I
O
U
, a center distance loss
L
d
i
s
and a width – height loss
L
a
s
p
, and ([pg. 10, line 1-2]: the penalty term of EIOU-loss comprises overlapping loss, central distance loss, width loss three parts, the formula is as follows) a calculation formula is as follows:
L
E
I
O
U
=
L
I
O
U
+
L
d
i
s
+
L
a
s
p
=
1
-
I
+
ρ
2
b
,
b
g
t
c
'
2
+
ρ
2
ω
,
ω
g
t
C
ω
2
+
ρ
2
(
h
,
h
g
t
)
C
h
2
(see equation on pg. 18, lines 26-27), wherein
b
and
b
g
t
respectively represent center points of a prediction box and a truth box;
ρ
represents an Euclidean distance between the two center points; c’ represents a diagonal distance of a smallest closure region that covers the prediction box and the truth box;
ω
g
t
and
h
g
t
respectively represent a length and width of the truth box;
ω
and
h
respectively represent a length and width of the prediction box; and
C
ω
and
C
h
respectively represent a width and height of a smallest external box that cover the truth box and the prediction box (see equation on pg. 18, lines 26-27, [pg. 10, paragraph 2]: the front two continues the method of CIOU-Loss, the width and high loss in order to make the convergence speed faster, directly setting the optimization target as the width height minimum difference of the real frame and the prediction frame, wherein, C and Ch are the width and height of the minimum external frame covering the real frame and the prediction frame.).
Claim(s) 5 is rejected under 35 U.S.C. 103 as being unpatentable over Huang (CN 114743084 A) and Jiang (CN 108596886 A) in view of Zhang (CN 110335238 A).
Regarding Claim 5, the Huang and Jiang combination combination teaches the method for detecting an insulator fault of a transmission line based on a USRNet and an improved MobileNet-SSD algorithm according to claim 1. Although Jiang teaches MobileNet-SSD for transmission line insulator fault detection, Jiang does not explicitly teach the remaining limitations of Claim 5. Zhang teaches wherein the constructed improved MobileNet-SSD detection model specifically comprises:
changing a structure of the MobileNet-SSD of the model, that is, adding eight different scales of convolutional layers after a last convolutional layer of MobileNetV1, wherein a shallow feature layer is used to detect a small target object, and a deep feature layer is used to detect a large target object; and extracting six different scales of effective feature maps from six of the layers through MobileNet-SSD, and performing multi-scale feature prediction, wherein resolutions of the effective feature maps are respectively 19 * 19, 10 * 10, 5 * 5, 3 * 3, 2 * 2, and 1 *1 ([pg. 3, paragraph 8]: the MobileNet is combined with the SSD algorithm, using MobileNet as the underlying network instead of VGG16 network of the SSD algorithm, after the Conv13 layer of MobileNet and adding 8 convolutional layer, extracting Conv11, Conv13, Conv14-2, Conv15-2, Conv16-2, Conv17-2 6 convolutional layer as the detection layer, is designed with a size layer. wherein the size of the Conv11 layer is 19 * 19 * 512, the size of the Conv13 layer is 10 * 10 * 1024 Conv14-2 layer size is 5 * 5 * 512 Conv15-2 layer size is 3 * 3 * 256 Conv16-2 layer size is 2 * 2 * 256 Conv17-2 layer size is 1 * 1 * 128, thereby obtaining the improved MobileNet-SSD algorithm).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified the Huang and Jiang combination to include the teachings of Zhang by including the improved MobileNet-SSD architecture using feature maps with different scales. Doing so would improve the accuracy of a defect detection model with faults of varying sizes.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JANICE VAZ whose telephone number is (703)756-4685. The examiner can normally be reached Monday-Friday 9:00-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JANICE E. VAZ/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights