DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 7 and 8 are objected to because of the following informalities:
Claim 7 recites “wherein the predefined spatial positions comprise one or more of: positions on one of the grid-lines adjacent to the node; and/or positions in one of the grid-fields adjacent to the node, wherein the predefined spatial positions comprise a predefined distance from the associated node.” Claim 7 refers to the grid-lines and grid-field when no prior reference has been made. Please correct the antecedent-basis issue.
Claim 8 should be amended as follows: “wherein the features exhibit a high optical contrast towards the front face of the substructure”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 11, and 12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 11 recites “wherein the tracking markers differ from each other in the optical appearance that is defined by the second set of features, particularly by the first set of features as well as by the second set of features.”
First, the last clause is redundant as it repeats “the second set of features.” Second, the term “particularly” seems to emphasize that “the first set of features” is a greater factor in differentiating the tracking markers than the second set of features without providing any clarity as to how the first set of features are a greater factor.
Examiner is interpreting claim 11 as follows: “wherein the tracking markers differ from each other in their respective optical appearances .”
Claim 12 recites “acquiring second features-set data that describes a position of a second set of features of the tracking marker within the plane of the image, particularly with respect to the first set of features…” The term “particularly” seems to emphasize that “the first set of features” is a greater factor in determining the position of the second set of features.
Examiner is interpreting claim 12 as follows: “acquiring second features-set data that describes a position of a second set of features of the tracking marker within the plane of the image…”
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite or similarly recite:
[a] determining grid data based on the first feature-set data, wherein the grid data describes a position of a grid pattern of the tracking marker within the plane of the image;
[b] determining identification data based on the first feature-set data and the second feature-set data, wherein the identification data describes an identity of the tracking marker;
[c] determining tracking data based on at least one of the first feature-set data, the second feature-set data and the off-plane data, wherein the tracking data describes a spatial position of the identified tracking marker.
Claim limitation [a], as drafted and under their broadest reasonable interpretation, recite a mathematical concept. (MPEP 2106.04(a)(2)(I) (see, e.g., Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (although the claims did not recite a particular mathematical formula, the court held “[w]ithout additional limitations, a process that employs mathematical algorithms to manipulate existing information to generate additional information is not patent eligible.”)). For example, determining grid data based on the first feature-set data involves identifying locations of the features using geometric calculations.
Claim limitations [b], as drafted and under their broadest reasonable interpretation, recite a mathematical concept and/or a mental process. (MPEP 2106.04(a)(2)(I) (see, e.g., Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (although the claims did not recite a particular mathematical formula, the court held “[w]ithout additional limitations, a process that employs mathematical algorithms to manipulate existing information to generate additional information is not patent eligible.”)). For example, determining identification data based on the first feature-set data and the second feature-set data, wherein the identification data describes an identity of the tracking marker includes identifying the information that is associated with that configuration of features.
Claim limitations [c], as drafted and under their broadest reasonable interpretation, recite a mathematical concept and/or a mental process. (MPEP 2106.04(a)(2)(I) (see, e.g., Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (although the claims did not recite a particular mathematical formula, the court held “[w]ithout additional limitations, a process that employs mathematical algorithms to manipulate existing information to generate additional information is not patent eligible.”)). For example, determining tracking data based on at least one of the first feature-set data, the second feature-set data and the off-plane data, wherein the tracking data describes a spatial position of the identified tracking marker requires performing various 3D calculations to determine the position of the marker.
The next question is to consider whether the claims integrate the judicial exception into a practical application. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception. (MPEP 2106.04(d)). In this case, some additional elements/steps to consider include acquiring first feature-set data that describes a position of a first set of features of the tracking marker within a plane of an image obtained via an optical camera; acquiring second features-set data that describes a position of a second set of features of the tracking marker within the plane of the image, particularly with respect to the first set of features; and acquiring off-plane data that describes a position of at least one feature offset from the plane, with respect to the first set of features and/or with respect to the second set of features.
Here, the judicial exception is not integrated into a practical application. These claim limitations recite insignificant extra-solution activity (i.e., pre-solution activity) that does not impose meaningfully limits on the claim. (MPEP 2106.04(d)(I), which also refers to MPEP 2106.05(g)).
The claims do not include additional elements/steps that are sufficient to amount to significantly more than the judicial exception. As explained above, these claim limitations recite insignificant extra-solution activity (i.e., pre-solution activity) that does not impose meaningfully limits on the claim. (MPEP 2106.04(d)(I), which also refers to MPEP 2106.05(g)).
Accordingly, claim 12 does not include patent-eligible subject matter.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-8, 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Appl. Publ. No. 2020/0163739 A1 (hereinafter “MESSINGER”) and Wang, Ben. "LFTag: A scalable visual fiducial system with low spatial frequency." 2020 2nd International Conference on Advances in Computer Technology, Information Science and Communications (CTISC). IEEE, 2020 (hereinafter “WANG”).
MESSINGER teaches a positioning marker that can be tracked during image-guided surgery. “During image guide surgery it is typically necessary to track objects used in the surgery, and/or elements of the patient undergoing the surgery.” ([0002]). Figure 3A shows a top view or a positional marker 18 and Figure 3B shows an exploded view. One particular use of the marker 18 is within an augmented reality system during surgery. (see, e.g., Figure 1).
With respect to claim 1, MESSINGER teaches a tracking marker for being positionally detected and tracked during a procedure ([0043]: Figure 1 schematically illustrates a “positioning marker 18” that can be used in an “augmented reality system 20…by a medical professional 22 in a medical procedure.”). The tracking marker includes a substantially planar substructure having a front face and defining a plane. ([0062] and [0068]: Marker 18 includes a base 80, cover 90, and a retainer 110 in which the base 80 is sandwiched between the cover 90 and the retainer 110. The cover 90 teaches a “substantially planar structure” having a “front face” defining a “plane.”). The tracking marker also includes a plurality of features disposed at and optically distinct from the front face. (The features are separate portions of
PNG
media_image1.png
559
873
media_image1.png
Greyscale
the retroreflective base 80 that are exposed by openings in the cover 90. See, e.g., [0064]: “When cover 90 is fastened to base 80, surface 92 of the cover mates with the retroreflective upper surface of base 80, and smaller circular openings 106 contact base 80, forming retroreflective circles 96, herein termed retroflectors 96.” See also [0066]: “[I]t will be appreciated that the retroreflective upper surface of base 80 is visible, through the openings, from many different directions. This visibility facilitates the ability of processor 26 to track marker 18 using camera 72 if there is relative motion between the camera and the marker, for example if professional 22 moves.”).
MESSINGER also teaches that the plurality of features include first and second sets of features defining, within the plane, a dimension and an orientation of a square grid pattern having an equidistant grid spacing, in which the second set of features are aligned with the grid pattern and (along with the first set of features) defines an optical appearance
PNG
media_image1.png
559
873
media_image1.png
Greyscale
[AltContent: textbox (Annotated Figure 3A of MESSINGER)][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector]that is specific for the tracking marker. ([0065]: “In one embodiment the centers of retroflectors 96 are formed on a square having a center on the z-axis. However, the spacing of the retroflectors on the square is configured to be non-symmetrical, so that processor 26 may determine a unique orientation of marker 18 from images of the retroflectors.”). NOTE: (1) As discussed above, Applicant’s appear to distinguish a “square grid pattern” of “nodes” and those features that are positioned at or adjacent to the nodes or absent altogether. Like Applicant’s square grid pattern (see Figure 2a of Applicant’s disclosure), MESSINGER teaches a square grid pattern in which the features (i.e., retroreflectors) are positioned at or adjacent to nodes of the square grid pattern. (2) Like Applicant’s embodiment (see Figure 2a of Applicant’s disclosure), MESSINGER teaches the second set of features being (generally) aligned with the grid pattern. (3) The four corner retroreflectors 96 of MESSINGER define a “dimension” and an “orientation” of the square grid pattern. (4) As to the “optical appearance” being specific for the “tracking marker,” the optical appearance is non-symmetrical so that the orientation may be determined. Applicant does not claim another tracking marker (unlike claim 12) so the appearance of MESSINGER’s marker is capable of being specific for that tracking marker.
MESSINGER also teaches at least one feature being aligned with the grid pattern and being disposed offset from the plane. ([0040]: “The retroreflectors are typically configured to be in a given plane, and in order to enhance the tracking provided by the marker, the marker may comprise a further retroreflector located in a plane different from the given plane.” (emphasis added). See also [0076]: “[A]dditional opening 130 and additional retroreflective surface 134 are configured to form a section 138 of the additional retroreflective surface that is in a different plane from the plane of sections of the retroreflective upper surface formed by openings 106….”) (emphasis added).
MESSINGER also teaches that the specific optical appearance is defined spatial positions of the individual features relative to associated nodes of the grid pattern. (See Annotated Figure 3A and retroreflectors 96 in which some are positioned at nodes and some are positioned adjacent to nodes (i.e., slightly offset). See also [0065]: “In one embodiment the centers of retroflectors 96 are formed on a square having a center on the z-axis. However, the spacing of the retroflectors on the square is configured to be non-symmetrical, so that processor 26 may determine a unique orientation of marker 18 from images of the retroflectors.”).
MESSINGER does not teach the grid pattern having nine nodes. MESSINGER also does not explicitly teach that the spatial positions of the features are selected from a limited set of predefined spatial positions, wherein distances for features being spaced from their associated node are the same for each feature of the tracking marker.
In the same field of endeavor, WANG teaches a scalable visual fiducial system that can be used with augmented reality and virtual reality systems. (Title and Abstract). Specifically, [AltContent: textbox (“Baseline” regions)][AltContent: textbox (“Data” regions)]WANG teaches “LFTag” which is “a visual fiducial system based on topological detection and relative position data encoding which optimizes data density within spatial frequency constraints. The marker is constructed to resolve rotational ambiguity, which combined with the robust geometric and topological false positive rejection, allows all marker bits to be used for data.” (Abstract). Comparing LFTag to existing state-of-the-art square binary markers, WANG notes that LFTag has “significant advantages in dictionary size and range.” (Abstract).
“LFTag is best described as a fiducial marker which performs data encoding with relative positioning of marker elements and detection through topological information.” (p.141, III. LFTag Design, bottom left column to top right column). The marker includes a black border, white background, and black internal regions. (Id., right column). “Two of the regions (also called ”baseline” regions) are larger than the others, and are at opposite sides of the top row of the marker. These points are used to resolve rotational ambiguity. The remaining regions are called ”data” regions, and each encode two bits in its relative location. Each data region has 4 possible locations, with the region centroids either shifting up or down, and left or right.” (Id). Annotated Figure 4a shows the four different positions in which, for each position, the center of the feature is offset an equal distance from the node (i.e., intersection of the grid lines). WANG further teaches that “[b]y encoding data with subtle shifts within data region positions…, LFTag minimizes the high spacial [sic] frequency content within the marker that is required for decoding [in another system, Topotag].” (Id). “The performance of LFTag at grazing angles matches the performance of square binary markers, as the low spatial frequency of the marker enables the markers to still be picked up with very limited vertical resolution.” (p.146, Discussion, right column).
The most discussed configuration in WANG is 3x3, which forms a square grid pattern having nine nodes. (See annotated Figure 4(a). However, LFTag is also scalable to other configurations. WANG also tested 4x4 and teaches that it is scalable to 8x8. (p.146, Discussion, right column). WANG discusses some areas for improvement in LFTag but also suggests modifications to address those areas. (p.146-147, VII. Future Work).
It would have been obvious to one having ordinary skill in the art to the grid pattern having nine nodes. One would have chosen to use a 3x3 grid pattern, thereby providing nine nodes, because the reduced number of nodes (compared to 5x5 in MESSINGER) would allow the features to be spaced apart even further, thereby reducing false positives, or allow a decrease in total size of the marker, thereby reducing the possibility of the marker colliding with other objects). It would also have been obvious to try another grid pattern because the 3x3 grid pattern is one of a finite number of identified, predictable solutions. There would have been a reasonable expectation of success as both MESSINGER and WANG teach that grid patterns can be used for tracking objects.
It would have also been obvious to one having ordinary skill in the art to modify the arrangement of the retroreflectors in MESSINGER such that the spatial positions of the features (i.e., retroreflectors) are selected from a limited set of predefined spatial positions, wherein distances for features being spaced from their associated node are the same for each feature of the tracking marker. More specifically, one skilled in the art would configure at least some of the retroreflectors in MESSINGER to have a spatial position that is a distance away from the node, as taught WANG, such that the distance from the node is the same for each of those retroreflectors. As WANG teaches, by subtly shifting the positions of the features, the marker can provide information while also be tracked with “very limited vertical resolution.” (p.146, Discussion, right column). There would have been a reasonable expectation of success as WANG teaches that markers having features with subtly shifted positions can still be identified.
NOTE: Examiner is interpreting “distances for features…spaced from their associated node” as including the distance measured from the center of the feature to the node. This definition is consistent with Applicant’s disclosure and, in fact, is the same definition used by Applicant when describing the embodiment of Figure 1a. “In the shown example, features 5 are spaced from their associated side-node 8 by a predefined distance s which is equal for every feature 5.” (page 17, last paragraph). Figure 1a illustrates that the “distance s” is measured from the center of the feature to the node.
With respect to claim 3, MESSINGER teaches that the dimension of the grid pattern is defined by four features defining respective corner nodes of the grid pattern. (See Figure 3a and the retroreflectors 96 at each of the four corners). NOTE: Claim 3 provides a list of limitations that are joined by “and/or….” Pursuant to the broadest reasonable interpretation of claim 3, the term “and/or” is being interpreted as “at least one…” such that the prior art must only teach one of the limitations.
With respect to claims 4 and 5, WANG teaches that wherein each of the features have one of two predefined sizes (claim 4). See Annotated Figure 4A of WANG in which the feature size can either be a larger border region or a smaller data region. “Two of the regions (also called ”baseline” regions) are larger than the others, and are at opposite sides of the top row of the marker. These points are used to resolve rotational ambiguity. The remaining regions are called ”data” regions, and each encode two bits in its relative location.” (p.141, III. LFTag Design, bottom left column to top right column).
It would have been obvious to one having ordinary skill in the art to configure the tracking marker to have features of one or two predefined sizes. One would be motivated to have at least two baseline features of a larger size, in order to determine the rotational position, and at least data regions of a smaller size in order to permit enough space to have different positions with respect to the node. There would have been a reasonable expectation of success as WANG teaches that features of different sizes can be used.
WANG also teaches that wherein the optical appearance is defined by the size of features of the first set of features as well as by the size of features of the second set of features (claim 5). The optical appearance would necessarily be based on the different sizes of the features.
With respect to claim 6, MESSINGER teaches that all of the features are of the same size. See Figure 3C showing the underside of the cover 90 in which the center opening is the same size of other openings that define the other features (i.e., retroreflectors). Moreover, Figure 4A and 4C show the center retroreflector having the same size as the other retroreflectors.
With respect to claim 7, the combined teachings of MESSINGER and WANG, as discussed above, teaches that the predefined spatial positions being positions on one of the grid-lines adjacent to the node and/or positions in one of the grid-fields adjacent to the node, wherein the predefined spatial positions comprise a predefined distance from the associated node. More specifically, WANG teaches that “[e]ach data region has 4 possible locations, with the region centroids either shifting up or down, and left or right.” (p.141, III. LFTag Design, bottom left column to top right column). See Annotated Figure 4a above. Examiner is interpreting these shifted positions as the centers being within the grid-fields adjacent to the nodes.
With respect to claim 8, MESSINGER teaches that the features are disk-shaped and are configured to reflect incident light (See Figure 3A of MESSINGER in which the holes 102 forming the retroreflectors are circular holes), wherein the features exhibit a high optical contrast towards the front face of the substructure ([0056]: “at least some retroreflected radiation” is enough to track the marker) , wherein the features include a retro-reflective coating. ([0061]: Retroreflectors may be made from “retroreflective paint.”)
With respect to claim 10, MESSINGER teaches that the at least one feature offset from the plane is disposed in an indentation or on a projection of the front face of the substructure. (For “indentation,” see perspective views Figures 3B and 3C and [0078]: “In the embodiment illustrated in FIGS. 3A, 3B, 3C section 138 is indented from the plane of openings 106….”; For “projection,” see side view in Figure 4C).
PNG
media_image1.png
559
873
media_image1.png
Greyscale
With respect to claim 11, MESSINGER teaches a tracking marker set ([0043]: Figure 1 schematically illustrates a “positioning marker 18” that can be used in an “augmented reality system 20…by a medical professional 22 in a medical procedure.”). The tracking marker includes a substantially planar substructure having a front face and defining a plane. ([0062] and [0068]: Marker 18 includes a base 80, cover 90, and a retainer 110 in which the base 80 is sandwiched between the cover 90 and the retainer 110. The cover 90 teaches a “substantially planar structure” having a “front face” defining a “plane.”). The tracking marker also includes a plurality of features disposed at and optically distinct from the front face. (The features are separate portions of the retroreflective base 80 that are exposed by openings in the cover 90. See, e.g., [0064]: “When cover 90 is fastened to base 80, surface 92 of the cover mates with the retroreflective upper surface of base 80, and smaller circular openings 106 contact base 80, forming retroreflective circles 96, herein termed retroflectors 96.” See also [0066]: “[I]t will be appreciated that the retroreflective upper surface of base 80 is visible, through the openings, from many different directions. This visibility facilitates the ability of processor 26 to track marker 18 using camera 72 if there is relative motion between the camera and the marker, for example if professional 22 moves.”).
MESSINGER also teaches that the plurality of features include first and second sets of features defining, within the plane, a dimension and an orientation of a square grid pattern having an equidistant grid spacing, in which the second set of features are aligned with the grid pattern and (along with the first set of features) defines an optical appearance
PNG
media_image1.png
559
873
media_image1.png
Greyscale
[AltContent: textbox (Annotated Figure 3A of MESSINGER)][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector][AltContent: connector]that is specific for the tracking marker. ([0065]: “In one embodiment the centers of retroflectors 96 are formed on a square having a center on the z-axis. However, the spacing of the retroflectors on the square is configured to be non-symmetrical, so that processor 26 may determine a unique orientation of marker 18 from images of the retroflectors.”). NOTE: (1) As discussed above, Applicant’s appear to distinguish a “square grid pattern” of “nodes” and those features that are positioned at or adjacent to the nodes or absent altogether. Like Applicant’s square grid pattern (see Figure 2a of Applicant’s disclosure), MESSINGER teaches a square grid pattern in which the features (i.e., retroreflectors) are positioned at or adjacent to nodes of the square grid pattern. (2) Like Applicant’s embodiment (see Figure 2a of Applicant’s disclosure), MESSINGER teaches the second set of features being (generally) aligned with the grid pattern. (3) The four corner retroreflectors 96 of MESSINGER define a “dimension” and an “orientation” of the square grid pattern. (4) As to the “optical appearance” being specific for the “tracking marker,” the optical appearance is non-symmetrical so that the orientation may be determined. Applicant does not claim another tracking marker (unlike claim 12) so the appearance of MESSINGER’s marker is capable of being specific for that tracking marker.
MESSINGER also teaches at least one feature being aligned with the grid pattern and being disposed offset from the plane. ([0040]: “The retroreflectors are typically configured to be in a given plane, and in order to enhance the tracking provided by the marker, the marker may comprise a further retroreflector located in a plane different from the given plane.” (emphasis added). See also [0076]: “[A]dditional opening 130 and additional retroreflective surface 134 are configured to form a section 138 of the additional retroreflective surface that is in a different plane from the plane of sections of the retroreflective upper surface formed by openings 106….”) (emphasis added).
MESSINGER also teaches that the specific optical appearance is defined spatial positions of the individual features relative to associated nodes of the grid pattern. (See Annotated Figure 3A and retroreflectors 96 in which some are positioned at nodes and some are positioned adjacent to nodes (i.e., slightly offset). See also [0065]: “In one embodiment the centers of retroflectors 96 are formed on a square having a center on the z-axis. However, the spacing of the retroflectors on the square is configured to be non-symmetrical, so that processor 26 may determine a unique orientation of marker 18 from images of the retroflectors.”).
MESSINGER does not teach the grid pattern having nine nodes. MESSINGER also does not explicitly teach that the spatial positions of the features are selected from a limited set of predefined spatial positions, wherein distances for features being spaced from their associated node are the same for each feature of the tracking marker. MESSINGER also does not teach a set including a plurality of tracking markers, wherein the tracking markers differ from each other in the optical appearance that is defined by the second set of features, particularly by the first set of features as well as by the second set of features, and which is specific for a respective tracking marker.
[AltContent: textbox (“Baseline” regions)][AltContent: textbox (“Data” regions)]In the same field of endeavor, WANG teaches a scalable visual fiducial system that can be used with augmented reality and virtual reality systems. (Title and Abstract). Specifically, WANG teaches “LFTag” which is “a visual fiducial system based on topological detection and relative position data encoding which optimizes data density within spatial frequency constraints. The marker is constructed to resolve rotational ambiguity, which combined with the robust geometric and topological false positive rejection, allows all marker bits to be used for data.” (Abstract). Comparing LFTag to existing state-of-the-art square binary markers, WANG notes that LFTag has “significant advantages in dictionary size and range.” (Abstract).
“LFTag is best described as a fiducial marker which performs data encoding with relative positioning of marker elements and detection through topological information.” (p.141, III. LFTag Design, bottom left column to top right column). The marker includes a black border, white background, and black internal regions. (Id., right column). “Two of the regions (also called ”baseline” regions) are larger than the others, and are at opposite sides of the top row of the marker. These points are used to resolve rotational ambiguity. The remaining regions are called ”data” regions, and each encode two bits in its relative location. Each data region has 4 possible locations, with the region centroids either shifting up or down, and left or right.” (Id). Annotated Figure 4a shows the four different positions in which, for each position, the center of the feature is offset an equal distance from the node (i.e., intersection of the grid lines). WANG further teaches that “[b]y encoding data with subtle shifts within data region positions…, LFTag minimizes the high spacial [sic] frequency content within the marker that is required for decoding [in another system, Topotag].” (Id). “The performance of LFTag at grazing angles matches the performance of square binary markers, as the low spatial frequency of the marker enables the markers to still be picked up with very limited vertical resolution.” (p.146, Discussion, right column).
The most discussed configuration in WANG is 3x3, which forms a square grid pattern having nine nodes. (See annotated Figure 4(a). However, LFTag is also scalable to other configurations. WANG also tested 4x4 and teaches that it is scalable to 8x8. (p.146, Discussion, right column). WANG discusses some areas for improvement in LFTag but also suggests modifications to address those areas. (p.146-147, VII. Future Work).
It would have been obvious to one having ordinary skill in the art to the grid pattern having nine nodes. One would have chosen to use a 3x3 grid pattern, thereby providing nine nodes, because the reduced number of nodes (compared to 5x5 in MESSINGER) would allow the features to be spaced apart even further, thereby reducing false positives, or allow a decrease in total size of the marker, thereby reducing the possibility of the marker colliding with other objects). It would also have been obvious to try another grid pattern because the 3x3 grid pattern is one of a finite number of identified, predictable solutions. There would have been a reasonable expectation of success as both MESSINGER and WANG teach that grid patterns can be used for tracking objects.
It would have also been obvious to one having ordinary skill in the art to modify the arrangement of the retroreflectors in MESSINGER such that the spatial positions of the features (i.e., retroreflectors) are selected from a limited set of predefined spatial positions, wherein distances for features being spaced from their associated node are the same for each feature of the tracking marker. More specifically, one skilled in the art would configure at least some of the retroreflectors in MESSINGER to have a spatial position that is a distance away from the node, as taught WANG, such that the distance from the node is the same for each of those retroreflectors. As WANG teaches, by subtly shifting the positions of the features, the marker can provide information while also be tracked with “very limited vertical resolution.” (p.146, Discussion, right column). There would have been a reasonable expectation of success as WANG teaches that markers having features with subtly shifted positions can still be identified.
NOTE: Examiner is interpreting “distances for features…spaced from their associated node” as including the distance measured from the center of the feature to the node. This definition is consistent with Applicant’s disclosure and, in fact, is the same definition used by Applicant when describing the embodiment of Figure 1a. “In the shown example, features 5 are spaced from their associated side-node 8 by a predefined distance s which is equal for every feature 5.” (page 17, last paragraph). Figure 1a illustrates that the “distance s” is measured from the center of the feature to the node.
It would have also been obvious to one having ordinary skill in the art to make or use at least two different tracking markers that differed in optical appearance. Medical procedures often track more than one object during the procedure. For example, optical markers can be attached to different surgical instruments and to one or more parts of the patient. (see, e.g., [0047] and [0083] of MESSINGER). One would have been motivated to make or use at least two different optical markers having different optical appearances in order to separately track the different objects to which the optical markers are attached. There would have been a reasonable expectation of success as WANG teaches that the features can be shifted to different positions in order to distinguish the optical markers while tracking.
With respect to claim 12, MESSINGER teaches a computer-implemented method of identifying and positionally tracking a tracking marker during a procedure. (see [0045] and [0046] describing processor identifying an ROI of a patient using the marker 18).
As discussed above with respect to claim 1, MESSINGER teaches an optical marker that includes first features, second features, and at least one feature offset from the plane. (The features are separate portions of the retroreflective base 80 that are exposed by openings in the cover 90. See, e.g., [0064]: “When cover 90 is fastened to base 80, surface 92 of the cover mates with the retroreflective upper surface of base 80, and smaller circular openings 106 contact base 80, forming retroreflective circles 96, herein termed retroflectors 96.” See also [0066]: “[I]t will be appreciated that the retroreflective upper surface of base 80 is visible, through the openings, from many different directions. This visibility facilitates the ability of processor 26 to track marker 18 using camera 72 if there is relative motion between the camera and the marker, for example if professional 22 moves.”). With respect to the off-plane feature, MESSINGER teaches that “in order to enhance the tracking provided by the marker, the marker may comprise a further retroreflector located in a plane different from the given plane.” ([0040]) (emphasis added). See also [0076]: “[A]dditional opening 130 and additional retroreflective surface 134 are configured to form a section 138 of the additional retroreflective surface that is in a different plane from the plane of sections of the retroreflective upper surface formed by openings 106….”) (emphasis added).
MESSINGER teaches acquiring images of first features, second features, and at least one feature that is off-plane. ([0038]: “Embodiments of the present invention provide a positioning marker which facilitates the registration referred to above, by enabling a processor coupled to the augmented reality assembly to track the marker, and to maintain the tracking even when large changes of angle, subtended by the marker to the assembly, occur.” See also [0081]: “By forming opening 130 as a frustum, retroreflector 138 [i.e., corresponding to off-plane data] is visible (as with retroflectors 96 of openings 98 [i.e., corresponding to first and second feature-set data]) from many different directions, so facilitating the ability of processor 26 to track marker 18 using camera 72.”). MESSINGER also teaches acquiring off-plane data that describes a position of at least one feature offset from the plane, with respect to the first set of features and/or with respect to the second set of features and determining tracking data based on at least one of the first feature-set data, the second feature-set data and the off-plane data, wherein the tracking data describes a spatial position of the identified tracking marker. (See [0074]: “To further facilitate tracking of marker 18, the marker comprises an additional opening 130 in cover 90, and an associated additional retroreflective surface 134 formed on an additional sheet 136.” See also [0046]: “In embodiments of the present invention professional 22 enables processor 26 to identify the ROI by locating positioning marker 18, described in more detail below, at a predefined location with respect to ROI 34, e.g., the ROI may be a predefined distance to the right and a predefined distance below marker 18. As is also described below, processor 26 is able to track the location of marker 18, and thus, since the marker is at a predefined location with respect to ROI 34, the processor is able to track the location of the ROI.” See also [0047] teaching that the positioning marker may be attached to a tool to track the tool).
However, MESSINGER does not explicitly teach the remaining limitations of claim 12. In the same field of endeavor, WANG teaches:
acquiring first feature-set data that describes a position of a first set of features of the tracking marker within a plane of an image obtained via an optical camera. After filtering the image data to determine candidate marker regions, WANG first assigns the largest regions as the baseline regions. See “Key Point Identification” section at paragraph beginning with “[f]irst, the regions are sorted by the inverted zeroth order image moment, after dilation by δ pixels. The two largest regions are picked as the baseline regions, denoted in arbitrary order by Ba and Bb.” (p.142, D. Key Point Identification, right column, second paragraph).
determining grid data based on the first feature-set data, wherein the grid data describes a position of a grid pattern of the tracking marker within the plane of the image. Knowing the two larger regions are the baseline regions, WANG then determines which of the two baseline regions each is. See “Key Point Identification” section at paragraph beginning with “[i]n order to disambiguate the order of these two regions, first a vector is found from Ba to Bb. Then, for every region in the marker, a vector is also calculated from Ba….” (p.142, D. Key Point Identification, right column, fourth paragraph).
acquiring second features-set data that describes a position of a second set of features of the tracking marker within the plane of the image, particularly with respect to the first set of features. After determining two corners of the baseline regions (called “key points”), WANG teaches finding the other two key points (i.e., corners) that are formed by the smaller, data regions. See “Key Point Identification” section at paragraph beginning with “[t]o locate the remaining two key points, a vector is drawn from K0 to K1, and for each remaining region, a vector is found from K0….” (p.142, D. Key Point Identification, right column, last paragraph). After determining the key points, WANG teaches confirming that all data points (other than the row with the baseline regions) are on the same side of the row with the baseline regions. “All data points are checked to ensure that all but n−2 data region centroids lie on the same side of the baseline vector. (This is also called the geometry constraint.)” (p.143, E. Geometry Filtering, left column, first paragraph).
determining identification data based on the first feature-set data and the second feature-set data, wherein the identification data describes an identity of the tracking marker. After determining a pose estimation based on the four key points (corners), WANG decodes the information from baseline and data regions. See “Tag Decoding” section beginning with “For each estimated pose, a decoding is also attempted….” (p.143, H. Tag Decoding, left column).
It would have also been obvious to one having ordinary skill in the art to use the image processing steps of WANG and analyze the first-features set data to identify the relative locations of the fixed corner features of MESSINGER and then use that information to identify the locations of the features in the second features-set data relative to the fixed corner features, as taught in WANG. Knowing the locations of the first and second features, one skilled in the art would then determining the identification data (i.e., decode) of the tracking marker as taught in WANG. One would be motivated to use this process because WANG’s use of “topological filtering produces roughly an order of magnitude less candidate regions compared to state of the art square marker detectors…,” thereby reducing workload and computation time (p.146, Discussion, left column). There would have been a reasonable expectation of success as WANG teaches the process may be used to determine position and identification data.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Appl. Publ. No. 2020/0163739 A1 (hereinafter “MESSINGER”) and Wang, Ben. "LFTag: A scalable visual fiducial system with low spatial frequency." 2020 2nd International Conference on Advances in Computer Technology, Information Science and Communications (CTISC). IEEE, 2020 as applied to claim 1 above, and further in view of U.S. Patent Appl. Publ. No. 2017/0086941 A1 (hereinafter “MARTI”) and Wang XY, Liu L, Guan MS, Liu Q, Zhao T, Li HB. The accuracy and learning curve of active and passive dynamic navigation-guided dental implant surgery: An in vitro study. Journal of Dentistry. 2022 Sep 1;124:104240 (hereinafter XIAO-YU).
With respect to claim 9, neither MESSINGER nor WANG teach that the each of the plurality of features comprises a light-emitting element.
In the same field of endeavor, MARTI teaches multiple designs for markers of optical tracking systems. (Title and [0003]). MARTI’s teaches similar configurations in which the one is an active configuration and the other is a passive configuration. (Compare Figures 1 and 2 (passive) to Figure 6 (active)). See also [0003]: “Traditional optical pose tracking systems comprise two cameras. They use triangulation to determine the three-dimensional (3D) position of light generating elements in space. These light generating elements may either be active: they transmit light (e.g. LEDs) or passive: they reflect light (e.g. reflective disks or spheres), or a combination of active and passive.”)
While MARTI teaches that active LEDs can be used to replace passive reflectors, XIAO-YU teaches that, for some circumstances such as dental implantation, active dynamic navigation is superior to the passive. “In the present study, the accuracy of active dynamic navigation system was greater than that of passive dynamic navigation system.” (Discussion). “In our study, the accuracy of M-PBR method was significantly better than that of F-PBR method for all the evaluated accuracy indicators (P < 0.01).” (Id). Moreover, the active system was quicker to learn. (Id).
It would have been obvious to one having ordinary skill in the art to substitute the retroreflectors of MESSINGER with the active LEDs as taught in MARTI. One would have chosen to use active LEDs over retroreflectors because, for certain procedures, active navigation systems perform better as taught in XIAO-YU. There would have been a reasonable expectation of success as MARTI teaches that similar configurations of tracking markers can use active LEDs instead of retroreflectors.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Appl. Publ. No. 2020/0163739 A1 (hereinafter “MESSINGER”) and Wang, Ben. "LFTag: A scalable visual fiducial system with low spatial frequency." 2020 2nd International Conference on Advances in Computer Technology, Information Science and Communications (CTISC). IEEE, 2020 as applied to claim 1 above, and further in view of Kalaitzakis, Michail, et al. "Fiducial markers for pose estimation: Overview, applications and experimental comparison of the artag, apriltag, aruco and stag markers." Journal of Intelligent & Robotic Systems 101.4 (2021): 71. (hereinafter “KALAITZAKIS”).
With respect to claim 9, MESSINGER does not teach using a monochrome mono-camera, that is configured to optically detect and distinguish the plurality of features from front face of the substructure. However, MESSINGER does teach using infra-red (single channel). Moreover, WANG teaches that the fiducial system can be used for “monocular pose estimation.” (Abstract and first paragraph).
Moreover, in the same field of endeavor, KALAITZAKIS shows that overwhelming majority of fiducial marker system use topological encoding (like MESSINGER and WANG) and are monochromatic. (See, e.g., Table 1).
It would have been obvious to one having ordinary skill in the art to use a monochrome mono-camera as taught in WANG and KALAITZAKIS that is configured to optically detect and distinguish the plurality of features from front face of the substructure. One would be motivated to use the monochrome mono-camera systems as it has been proven to be successful in tracking planar markers. There would have been a reasonable expectation of success as WANG and KALAITZAKIS demonstrate that monochrome mono-camera systems can be used effectively to track planar markers.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASON P GROSS whose telephone number is (571)272-1386. The examiner can normally be reached Monday-Friday 9:00-5:00CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne M. Kozak can be reached at (571) 270-5284. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JASON P GROSS/ Examiner, Art Unit 3797
/SERKAN AKAR/ Primary Examiner, Art Unit 3797